Garage security and convenience features

ABSTRACT

A garage door is controlled to open using an audio/video (A/V) recording and communication device. The device detects a vehicle within an area about the garage door and receives, using a camera, image data representative of an object associated with the vehicle. The device also compares the image data representative of the object associated with the vehicle with previously stored image data and identifies the object based on the comparing of the image data representative of the object with the previously stored image data. The device also authenticates an electronic device within the vehicle that is associated with the object by wirelessly communicating with the electronic device and determines, based at least in part on the authenticating of the electronic device within the vehicle, that the vehicle is an authorized vehicle. The device also transmits an actuation command to the garage door to cause the garage door to open.

BACKGROUND

Home security is a concern for many homeowners and renters. Thoseseeking to protect or monitor their homes often wish to have video andaudio communications with visitors, for example, those visiting anexternal door or entryway. Audio/video (A/V) recording and communicationdevices, such as doorbells, provide this functionality, and can also aidin crime detection and prevention. For example, audio and/or videocaptured by an A/V recording and communication device can be uploaded tothe cloud and recorded on a remote server. Subsequent review of the A/Vfootage can aid law enforcement in capturing perpetrators of homeburglaries and other crimes. Further, the presence of one or more A/Vrecording and communication devices on the exterior of a home, such as adoorbell unit at the entrance to the home, acts as a powerful deterrentagainst would-be burglars.

BRIEF DESCRIPTION OF THE DRAWINGS

The various embodiments of the present garage security and conveniencefeatures now will be discussed in detail with an emphasis onhighlighting the advantageous features. These embodiments depict thenovel and non-obvious garage security and convenience features shown inthe accompanying drawings, which are for illustrative purposes only.These drawings include the following figures, in which like numeralsindicate like parts:

FIG. 1A is a schematic diagram of an example of A/V recording andcommunication devices used to monitor a garage and an area about agarage, according to various aspects of the present disclosure;

FIGS. 1B and 1C are schematic diagrams of examples of an A/V recordingand communication device used to monitor the movement of objects in aroom, according to various aspects of the present disclosure;

FIGS. 1D-1F are schematic diagrams of examples of a front view of agarage with a garage door in an open state, a partially open state, anda closed state, respectively, according to various aspects of thepresent disclosure;

FIG. 2 is a functional block diagram illustrating a system forcommunicating in a network according to various aspects of the presentdisclosure;

FIG. 3 is a functional block diagram of an A/V recording andcommunication device according to various aspects of the presentdisclosure;

FIG. 4 is a functional block diagram illustrating one example embodimentof an A/V recording and communication device according to variousaspects of the present disclosure;

FIG. 5 is a functional block diagram illustrating one example embodimentof a backend device according to various aspects of the presentdisclosure;

FIG. 6A is a functional block diagram illustrating one exampleembodiment of a client device according to various aspects of thepresent disclosure;

FIG. 6B is a functional block diagram illustrating one exampleembodiment of a client automobile device according to various aspects ofthe present disclosure;

FIG. 7 is a functional block diagram illustrating one example embodimentof a smart-home hub device according to various aspects of the presentdisclosure;

FIG. 8 is a flowchart illustrating an example process for detecting avehicle and opening a garage door, according to various aspects of thepresent disclosure;

FIG. 9 is a flowchart illustrating an example process for transmittingaudible information to a speaker, according to various aspects of thepresent disclosure;

FIGS. 10 and 11 are flowcharts illustrating example processes fordetecting a vehicle and closing a garage door, according to variousaspects of the present disclosure;

FIG. 12 is a flowchart illustrating an example process for monitoringthe movement of objects inside of a room, according to various aspectsof the present disclosure;

FIG. 13 is a flowchart illustrating an example process for providingguest authorization to access a garage, according to various aspects ofthe present disclosure;

FIG. 14 is a flowchart illustrating an example process for providingscannable codes for authorizing access to a garage, according to variousaspects of the present disclosure;

FIG. 15 is a flowchart illustrating an example process for providing adelay in closing a garage door to allow a person to exit a garage,according to various aspects of the present disclosure;

FIG. 16 is a flowchart illustrating an example process for changing thestate of a garage door based on vehicle status information, according tovarious aspects of the present disclosure;

FIG. 17 is a flowchart illustrating an example process for using avehicle location to determine whether a garage door should be closed,according to various aspects of the present disclosure;

FIG. 18 is a flowchart illustrating an example process for controlling agarage door with a voice command, according to various aspects of thepresent disclosure;

FIG. 19 is a flowchart illustrating an example process for authorizing asingle button press to change a state of a garage door, according tovarious aspects of the present disclosure;

FIG. 20 is a flowchart illustrating an example process for using adistress code that opens a garage door and alerts a security monitoringservice, according to various aspects of the present disclosure;

FIG. 21 is a flowchart illustrating an example process for capturing animage in an opening of a garage, according to various aspects of thepresent disclosure;

FIG. 22 is a flowchart illustrating an example process for transmittinga reminder message to complete a to-do list item based on vehiclelocation information, according to various aspects of the presentdisclosure;

FIG. 23 is a flowchart illustrating an example process for determiningwhether to change a state of a garage door based on atmospheric datameasured in a garage, according to various aspects of the presentdisclosure;

FIG. 24 is a functional block diagram of a client device on which thepresent embodiments may be implemented according to various aspects ofthe present disclosure; and

FIG. 25 is a functional block diagram of a general-purpose computingsystem on which the present embodiments may be implemented according tovarious aspects of present disclosure.

DETAILED DESCRIPTION

The various embodiments of the present garage security and conveniencefeatures have several features, no single one of which is solelyresponsible for their desirable attributes. Without limiting the scopeof the present embodiments as expressed by the claims that follow, theirmore prominent features now will be discussed briefly. After consideringthis discussion, and particularly after reading the rest of the DetailedDescription, one will understand how the features of the presentembodiments provide the advantages described herein.

Access points to buildings can be more vulnerable to intruders thanother parts of a home. For example, a door is more likely to be breachedfrom a break-in than a wall. However, access points should also beeasily accessible to those authorized to enter a building. For example,an authorized person may have a key to unlock a locked door and access abuilding. Accordingly, access points to a building should have featuresthat balance providing convenient and easy access to authorized parties,while also providing security features that help prevent access to abuilding by unauthorized parties. Embodiments of the present disclosureprovide convenient yet safe access to a building and are describedherein. For example, various embodiments described herein relate toaccessing a garage through a garage door in safe and convenient ways.

In an example embodiment, an A/V recording and communication device isattached to the outside of a garage, such that a field of view of theA/V recording and communication device includes an area about the garage(e.g., in front of, around, near, next to). The area about the garagemay include a driveway leading to the garage and the garage's garagedoor. Thus, when a vehicle is approaching the garage or is otherwise inthe driveway, the vehicle will be in the field of view of the A/Vrecording and communication device. The A/V recording and communicationdevice captures image data. From this image data, the A/V recording andcommunication device or another component of the system detects that avehicle is approaching and/or present. The system may also determinewhether the vehicle is authorized to access the garage. Image data maybe, for example, captured by a camera. Such image data is not limited toimages captured by visible light cameras. Image data may, for example,be captured by night vision cameras, infrared cameras, motion sensors,depth sensors, and/or by any other type of sensor or camera as describedherein.

The determination whether the vehicle is authorized to access the garagemay be made in various ways. For example, the image data may be analyzedby the A/V recording and communication device or another component ofthe system to identify the vehicle. For example, identification of thevehicle may include a determination of a type of the vehicle (includingyear, make, and/or model), a color of vehicle, a license plate number, alicense plate state of issue, a license plate renewal date, customfeatures of a vehicle, any other aspect of a vehicle that may bedetermined from images captured of a vehicle by the A/V recording andcommunication device, or any combination thereof. If a vehicle isdetected approaching a garage and that vehicle has attributes that matchthe known attributes of an authorized vehicle, then an actuation commandis transmitted to an actuator of the garage door to open it. In anotherexample, a computing device in a vehicle may be able to communicate withthe A/V recording and communication device. In this way, the vehiclecomputing device communicates authorization information to the A/Vrecording and communication device. If the A/V recording andcommunication device recognizes the authorization information, then theactuation command is transmitted to the actuator of the garage door toopen the garage door.

In another example embodiment, the A/V recording and communicationdevice is used to determine when a garage door should be closed. Forexample, the A/V recording and communication device may determine fromimage data that a vehicle has left the garage and the area about thegarage (e.g., the driveway). Once the vehicle is gone, if the garagedoor is still open, an actuation command is transmitted to the actuatorof the garage door to close the garage door.

Various embodiments are described herein for improving the convenienceof accessing a building and improving the security in and around thebuilding, including access points of a building (e.g., a garage door).For example, a garage may have an A/V recording and communication deviceattached to the outside, such that a field of view of the A/V recordingand communication device includes an area about the garage. The areaabout the garage may include a driveway leading to the garage and thegarage's garage door. Thus, when a vehicle is approaching the garage oris otherwise in the driveway, the vehicle will be in the field of viewof the A/V recording and communication device. The A/V recording andcommunication device captures image data. Based on the captured imagedata, the A/V recording and communication device or another component ofthe system determines that the vehicle is approaching the garage or isotherwise in the driveway. This image data is used to determine whetherthe vehicle is authorized to access the garage. If the vehicle isauthorized to access the garage, an actuation command to open the garagedoor is transmitted.

In various embodiments, other methods for determining that a vehicle isapproaching a garage may be used instead of or in addition to using theimage data captured by the A/V recording and communication device. Forexample, geolocation may be used to determine the location of anauthorized vehicle. A vehicle electronic device may be equipped with ageolocation module (e.g., a global positioning system (GPS) unit) sothat a specific vehicle's location may be determined. Other clientdevices may also have a geolocation module. These various client devicesmay determine their geolocation using satellite-based positioning system(e.g., Global Positioning System, Global Navigation Satellite System,BeiDou, Galileo, etc.), a Wi-Fi positioning system (WFPS), or anInternet Protocol (IP) based geolocation lookup service, among otherpossible geolocation determination techniques. The location of a vehicleis therefore be used to determine if a vehicle is approaching aparticular garage. Geolocation of a vehicle and the A/V recording andcommunication device may also be used in combination to determinewhether a vehicle is approaching a garage and determine whether thatvehicle is authorized to enter the garage. For example, the A/Vrecording and communication device may recognize that a vehicle isapproaching a garage in a driveway. The A/V recording and communicationdevice or another component of the system determines, using geolocation,whether any vehicles authorized to enter the garage are in the generallocation of the garage. If so, the vehicle is authorized to enter thegarage, and an actuation command may be transmitted to open the garagedoor.

In other embodiments, client devices associated with an authorizedvehicle or client/user may communicate authorization information so thatthe garage will be opened. Such communications may occur wirelessly(e.g., through near field communications (NFC), Bluetooth Low Energy(BLE), radio-frequency identification (RFID), or other types of wirelesscommunications devices). In this way, a client device communicateswirelessly that it is authorized to enter a garage and cause anactuation command to be transmitted to open a garage door. Suchcommunication methods may also be used, alone or in combination withidentification of a vehicle from image data captured by an A/V recordingand communication device, to determine the presence of a vehicle orperson in an area about a garage for which the garage door may beopened.

The A/V recording and communication device may also be used to determinewhen a garage door should be closed. For example, the A/V recording andcommunication device determines from image data that a vehicle has leftthe garage and the area about the garage (e.g., the driveway). Once thevehicle is gone, if the garage door is still open, an actuation commandmay be transmitted to the actuator of the garage door to close thegarage door. Additionally, wireless communication devices may be used inaddition to or instead of image data from an A/V recording andcommunication device to determine that a vehicle has left a garage.

In some implementations, an authorized client device may also be used togenerate or send scannable codes that allow a visitor to access agarage. For example, a scannable visual code, such as a QR code, may begenerated or sent to a visitor to provide limited access to a garage forthat visitor. The visitor may receive or retrieve the scannable visualcode on an electronic device, then scan the code at a specializedscanner at the garage to allow access. In some implementations, thescannable code may be presented to a camera of an A/V recording andcommunication device. The scannable code may then be determined based onimage data captured by the A/V recording and communication device. Insome implementations, a visitor's electronic device may be equipped withwireless communication capabilities. For example, a smart phone or otherdevice may have BLE, NFC, RFID, or other wireless communicationcapabilities. In such an embodiment, the scannable code may bewirelessly communicated rather than via scanning a visual code such as aQR code. These scannable codes may give a visitor limited access to thegarage. For example, the scannable code may permit a visitor to accessthe garage a predetermined number of times. In another example, thescannable code may permit a visitor to access the garage during apredetermined amount of time, such as within a certain amount of timefrom when the visitor receives the scannable code or in a certainpredetermined time window. The parameters of the limited access may beset by the authorized party that generated or sent the scannable code.

An A/V recording and communication device in a garage may have a fieldof view such that objects in a garage may be identified from image datacaptured by the A/V recording and communication device. Such objects mayinclude vehicles, boxes, power tools, appliances, lawn mowers, skis, orany other object. Accordingly, the A/V recording and communicationdevice may be used over time to track when objects in the garage aremoved and record images or video of such events. In someimplementations, the system may determine whether an authorized party ispresent when the object is moved. If an authorized party is not presentwhen the A/V recording and communication device detects movement withinthe garage of a tracked object (or otherwise detects movement), the A/Vrecording and communication device may capture video and/or audio of themovement. In this way, authorized parties may be alerted to the movementin real time and/or review images and video of the movement at a latertime to determine what happened in the garage. Since objects in thegarage may be identified, the system may also keep an inventory of whatis stored in the garage. When an object is removed from the garage, theinventory may reflect that the item is no longer there. Such aninventory may be checked by an authorized party when they are lookingfor an object to determine if it is still in the garage.

The image data captured by the A/V recording and communication devicemay also be used to assist a party in parking a vehicle (e.g., cars,trucks, mowers, all-terrain vehicles, snowmobiles, farm equipment)inside a garage. For example, the A/V recording and communication devicemay monitor the location of a car as it pulls into the garage. As thecar pulls in, the A/V recording and communication device may determinewhether the car is fully in the garage so that the garage door may beclosed. If it is not, an audio message may be broadcast by the A/Vrecording and communication device or transmitted to a speaker in thecar to advise the driver that the car is not completely in the garage(e.g., “Please pull forward, the vehicle is not clear of the garagedoor.”). The A/V recording and communication device may also monitor thevehicle to ensure that the vehicle does not hit another object orvehicle in the garage or a wall of the garage. For example, the A/Vrecording and communication device may broadcast or transmit an audiomessage indicating that the car is getting close to a wall (e.g., “Becareful, you are close to the wall.”). In another implementation, theA/V recording and communication device may be used to make sure objectsand vehicles are stored in a predesignated location within the garage.Some people store many things in their garage, and their vehicles andobjects may not fit (or may not fit well to allow access to everything)if things are not arranged within the garage properly. The A/V recordingand communication device may be used to ensure that things are placedwhere they are supposed to be. For example, if someone leaves a lawnmower in a space where a car is supposed to be stored, an audio messagemay be broadcast by the A/V recording and communication device ortransmitted to another device that the lawn mower should be moved to adifferent location within the garage.

The remaining detailed description describes the present embodimentswith reference to the drawings. In the drawings, reference numbers labelelements of the present embodiments. These reference numbers arereproduced below in connection with the discussion of the correspondingdrawing features.

FIG. 1A is a schematic diagram of an example of A/V recording andcommunication devices used to monitor a garage 114 and an area about agarage, according to various aspects of the present disclosure. FIG. 1Ashows a building 138 from a top view. The building 138 includes a livingarea 102 and the garage 114. The garage 114 has an opening 116 where agarage door may open and close, allowing ingress and egress to and fromthe garage 114 for vehicles, people, animals, objects, etc.

A driveway 132 provides a path into the garage 114 and to a road 134. Afirst car 122 is in the garage 114, a second car 130 is in the driveway132, and a third car 136 is in the road 134. Each of the first car 122,the second car 130, and the third car 136 may include a client vehicledevice, such as a client vehicle device 230 described herein and shownin at least FIGS. 2 and 6B. A person 112 with a client device is in thedriveway 132, a person 124 with a client device is in the garage 114,and a person 110 with a client device is in the living area 102. Themobile client devices may be, for example, smartphones, laptops, orother computing devices such as the client devices 214 and 216 describedherein and shown in at least FIGS. 2 and 6A. A smart-home hub device 108is in the living area 102. An example of the smart-home hub device 108may be, for example, a smart-home hub device such as the smart-home hubdevice 202 described herein and shown in at least FIGS. 2 and 7.

Attached in and on the building 138 are multiple A/V recording andcommunication devices. A security camera 118 is located in the garage114, a light camera 126 (e.g., a floodlight camera, a spotlight camera,etc.) is located on the outside of the building 138 on the garage 114,and a video doorbell 104 (e.g., a wall powered and/or battery poweredvideo doorbell) is located at a front door of the living area 102 of thebuilding 138. Each of the security camera 118, the light camera 126, andthe video doorbell 104 are A/V recording and communication devices asdescribed herein, such as the A/V recording and communication devices210 shown in FIGS. 2-4 or any other devices capable of recording audiodata and/or image data. In various embodiments, those devices may beinterchanged, and there may be more, less, and/or different A/Vrecording and communication devices than are shown in FIG. 1A.

The light camera 126 has a field of view 128 in which a camera of thelight camera 126 may capture image data. That image data may be stillframes of image data, or may be a plurality of frames of image data suchas a video. In FIG. 1A, the vehicle 130 and the person 112 are in thefield of view 128 of the light camera 126. As described herein, thevehicle 130 may be identified based on image data captured by the lightcamera 126. After identifying the vehicle 130, the light camera 126(and/or another device in the system that is in communication with thelight camera 126) may determine whether to change the state of thegarage door of the garage 114 (e.g., open or close the garage door). Forexample, if the vehicle 130 is pulling into the driveway 132, the garagedoor may be opened so the vehicle 130 may pull through the opening 116of the garage 114. If the vehicle 130 is pulling out of the driveway132, the garage door may be closed. If the vehicle 130 is parked orstationary, the garage door may remain unchanged in its current state.

The person 112 is also in the driveway 132 and in the field of view ofthe light camera 126. The system may also take into account the presenceof the person 112 when determining whether to change the state of thegarage door. Image data captured by the light camera 126 may also bestored or sent to various devices to alert a client of what is happeningin their driveway. For example, the system may determine, from imagedata captured by the light camera 126, that the vehicle 130 and/or theperson 112 moved onto the driveway. In response, the system may savethat image data or begin streaming that image data to a client deviceassociated with the light camera 126 (e.g., the smart-home hub device108, the client devices of the person 110 and/or 124, a client vehicledevice). In this way, a client may see what is going on in theirdriveway. The system may be configured to store and/or stream movementof vehicles or people that are outside the driveway 132 as well (e.g.,the vehicle 136 in the road 134). Alternatively, the system may beconfigured to ignore movement of vehicles and/or people outside thedriveway 132 (e.g., the vehicle in the road 134).

The video doorbell 104 has a field of view 106. The field of view 106 ofthe video doorbell 104 also covers at least a portion of the driveway132. Accordingly, image data captured by the video doorbell 104 may alsobe used to implement various garage safety and convenience features asdescribed herein. For example, the image data captured by the videodoorbell 104 and/or the light camera 126 may be used to identify vehicleinformation about the vehicle 130. The image data captured by the videodoorbell 104 and/or the light camera 126 may also include informationencoded into scannable visual codes, such as QR codes. For example, theperson 112 may have a QR code displayed on their client device. Theperson 112 may then present the QR code to the video doorbell 104 and/orthe light camera 126. The image data captured will then include the QRcode, which is read to determine whether the garage door should beopened or closed for the person 112 as described herein. A level ofcertainty of identifying vehicle information (e.g., of the vehicle 130)may also be increased by capturing image data of a vehicle from morethan one A/V recording and communication device (e.g., the videodoorbell 104 and the light camera 126).

Inside the garage 114 is the security camera 118. A field of view 120 ofthe security camera 118 demonstrates that the security camera 118 maycapture images data of what is in the garage 114, including the vehicle122 and the person 124. The opening 116 is also in the field of view 120of the security camera 118. In this way, image data captured by thesecurity camera 118 may be used to determine if the garage door is openor closed, determine if anything or anyone is in the opening 116 whenthe garage door is open, and/or determine if anything or anyone is inthe driveway 132 when the garage door is open.

The image data captured by the security camera 118 may also be used todetermine other things. For example, the image data captured by thesecurity camera 118 may be used to determine when to open the garagedoor. The person 124 may have come from the living area 102 and may getin the vehicle 122. Image data captured by the security camera 118 couldshow this behavior and indicate that the person wants to leave, thus anactuation command may be transmitted to open the garage door. Furtherexamples for how image data may be used to determine when a garage dooris opened are described herein, including with respect to FIG. 8discussed below. The image data may also indicate that a car, such asthe vehicle 122 is safely parked within the garage 114, and the garagedoor may therefore be closed. Further examples for how image data may beused to determine when a garage door is closed are described herein,including with respect to FIGS. 10 and 11 discussed below. The imagedata captured by the security camera 118 may also be used to provideinstructions for parking a vehicle in the garage 114, such as thevehicle 122. Further examples for how image data may be used to provideinstructions or take actions for placing an object or vehicle in thegarage are described herein, including with respect to FIGS. 9 and 15discussed below. The image data captured by the security camera 118 mayalso be used to determine how objects, vehicles, and/or people havemoved in and out of the garage 114. Examples may include keeping aninventory of objects in a garage, alerting a client device of suspiciousmovement and/or a missing object in the garage, and/or recognize newobjects being put in a garage. Further examples of how image data isused to track movement of people, objects, and vehicles move in a garageare described herein, including with respect to FIG. 12.

In the living area 102 of the building 138 is the person 110 and thesmart-home hub device 108. The person 110 may be able to give voicecommands to the smart-home hub device 108, such as instructing thegarage door to be opened or closed. The smart-home hub device 108 isconnected (either through a wired or wireless connection) to acontroller of an actuator of the garage door. In this way, the person110 may control the garage door from inside the living area 102. Theperson 110 may also control the garage door with their client device,which may transmit an actuation command to the controller of theactuator of the garage door. That actuation command may be routedthrough the smart-home hub device 108, a wireless network, a cellularnetwork, and/or any other device. The actuation command may also be sentdirectly to the controller of the actuator of the garage door. Theperson 112 and the person 124 may similarly communicate with and/orcontrol the garage door according to the various embodiments describedherein.

Client devices (e.g., the client devices of the persons 110, 112, and/or124), the smart-home hub device 108, the security camera 118, the videodoorbell 104, the light camera, and client vehicle devices (e.g., clientvehicle devices of the vehicles 122, 130, and/or 136) may all bewirelessly connected. Accordingly, functions described herein asoccurring on a particular device may be performed on any one of thedevices. For example, while image data of objects in the garage 114 iscaptured by the security camera 118, the smart-home hub device 108 maykeep track of an inventory in the garage and determine if/when an alertshould be sent to a client device about an object leaving the garage 114or some other suspicious movement. Information may be also stored on aremote server or servers not shown that are in communication with anyand/or all of the devices shown in FIG. 1A. Methods described herein mayalso be performed on a remote server or servers not shown that are incommunication with any and/or all of the devices shown in FIG. 1A.

FIGS. 1B and 1C are schematic diagrams of examples of an A/V recordingand communication device used to monitor the movement of objects in aroom, according to various aspects of the present disclosure. Althoughobjects are shown and discussed with respect to FIGS. 1B and 1C, peopleand vehicles may also be monitored using similar methods as describedherein. FIGS. 1B and 1C shows a room with a security camera 142 mountedin the corner. In various embodiments, the security camera may be anytype of A/V recording and communication device.

In various embodiments, more than one A/V recording and communicationdevice may be mounted or otherwise used in a room. Multiple A/Vrecording and communication devices may be used to better capture imagedata of all portions of a room. Multiple A/V recording and communicationdevices may also be used to better identify objects and determine exactlocations of objects in a room. For example, image data captured by anA/V recording and communication device may be used to determine depthdata (e.g., how far away an object is from the A/V recording andcommunication device). By having multiple A/V recording andcommunication devices in the room, that depth data may be used totriangulate precise locations of objects in the room and/or betterdetermine the shape of objects in the room. A/V recording andcommunication devices may also be used to watch different parts of aroom. For example, if the room in FIGS. 1B and 1C had multiple cameras,one camera may be oriented such that the field of view captures shelves140, while a second camera is oriented such that the field of viewcaptures a floor 141 of the room. In this way, the A/V recording andcommunication devices may better keep inventory, track changes, watchfor movement, etc. as described herein.

The security camera 142 is oriented such that its field of view capturesthe entire room of FIGS. 1B and 1C. The room in FIGS. 1B and 1Cspecifically is the inside of a garage. Although FIGS. 1B and 1C show agarage, the objects of any room may be monitored according to thevarious embodiments described herein. Additionally, a security cameramay be able to monitor objects in outside areas (e.g., a yard, a parkinglot, a sports field, a park, a driveway, farm land) or in structuresthat may not be classified as a room or garage (e.g., a storage shedlacking one or more walls). The field of view of the security camera 142therefore captures the shelves 140 and any objects stored thereon, aswell as the floor 141 of the garage and any objects stored thereon. Thefield of view of the security camera 142 also captures an openingopposite the shelves in the garage where a garage door may open andshut. In this way, the security camera 142 may also be used to determinewhat state the garage door is in (e.g., open, closed, somewhere inbetween). The security camera 142 may also be used to capture image dataof anything outside the garage when the garage door is at leastpartially open. The image data captured by the security camera 142 maybe used to perform any of the methods described herein.

FIG. 1B shows the inside of a garage at a first time. FIG. 1C shows aninside of a garage at a second time. In FIG. 1B, boxes 144(a)-(d) arestored on the shelves 140. A crate 146 and a barrel 148 are stored onthe floor 141 of the garage. The presence of these objects aredetermined to be within the garage, and an inventory of the objects isstored in a memory of the security camera 142 or another device (e.g.,the smart-home hub device 108 of FIG. 1, a server). The specificlocation of the objects within the garage is also determined from imagedata captured by the security camera 142. The inventory may then alsoreflect where in the garage each of the objects is located.

FIG. 1C shows the inside of a garage at a second time. Comparing FIG. 1Cto FIG. 1B, the boxes 144(a), 144(b), and 144(d) remain on the shelves140. However, the box 144(c) has been removed from the shelves 140. Thismay be determined from image data captured by the security camera 142 atthe first time (FIG. 1B) and the second time (FIG. 1C). That removal maybe indicated in the inventory kept of the objects in the garage. Inaddition, the security camera 142 may capture image data relating to themovement of the box 144(c) itself. For example, if the security camera142 captures motion within the garage and/or movement of the box 144(c),image data of that movement is captured and stored in a memory of thesecurity camera 142 and/or another device. The image data of the motionwithin the garage and/or movement of the box 144(c) may also be streamedto another device. For example, video of the motion and/or movement maybe streamed to a client device or client vehicle device as describedherein (e.g., one of the devices shown in 6A or 6B). In this way, if anitem goes missing, an owner of the item may review or see the video tobe alerted of a theft or be able to investigate the theft.

Comparing FIGS. 1B and 1C, the box 144(d) has been moved. Although thebox 144(d) was not taken out of the garage, the security camera 142still captures image data of the movement of the box 144(d). Similar tothe movement of the box 144(c), the image data captured of the movementof the box 144(d) may be stored in the memory of an electronic deviceand/or may be streamed to an electronic device. Additionally, theinventory of the objects within the garage is adjusted to reflect thenew location of the box 144(d). For example, a two or three-dimensionalgrid may be used to identify locations of objects on a shelving unit, onthe floor, or anywhere else an object may be located. The coordinatesindicating the location of the box 144(d) within the garage according tothe grid system are updated after the box 144(d) moves from its locationin FIG. 1B to a new location in FIG. 1C.

Similar to the boxes 144(a) and 144(b), the crate 146 and the barrel 148on the floor 141 do not move between the first time reflected in FIG. 1Band the second time reflected in FIG. 1C. Accordingly, the inventoryregarding the crate 146 and the barrel 148 would not be changed based onimage data captured by the security camera 142 from the first time tothe second time. However, a crate 150 has been added to the floor 141 inthe time between FIGS. 1B and 1C. As with the movement of the boxes144(c) and 144(d), the security camera 142 captures movement relating tothe crate 150 being placed on the floor 141. That image data may bestored and/or streamed to an electronic device. The inventory kept bythe system is also updated to reflect that the new crate 150 has beenadded to the garage, and the specific location/coordinates of the crate150 within the garage may also be indicated in the inventory.

The image data from the security camera 142 may also be used for otherpurposes. For example, the system may determine that the crate 150 hasbeen placed where a vehicle would normally park. In response to thisdetermination, an alert (e.g., a text message, an email) may be sent toa client device, client vehicle device, or smart-home hub device thatthe crate 150 is in the way of parking a vehicle in the garage. An audiomessage may also be played alerting a person that the crate 150 shouldnot be placed in that location on the floor. For example, the securitycamera 142 may include a speaker through which the audio message may beplayed (e.g., “Please move the crate to the shelving unit or place itnear the opposite wall.”). The audio message may also be transmitted toand played by another device, such as a client vehicle device (e.g.,“Warning, there is an object obstructing the parking place in yourgarage.”), a smart-home hub device (e.g., “An object in the garageshould be moved before the driver of your vehicle returns home.”), orany other electronic device.

A system may make a determination from image data whether the movement(particularly when objects are removed from the garage) of an object isauthorized. If the movement of an object is not authorized, alerts maybe sent to a client device and/or client vehicle device. The system mayincorporate location information of client devices and/or client vehicledevices to determine whether object movement is authorized or not. Forexample, if a client device is located in the garage when an object istaken out of the garage, the movement of that object may be classifiedas authorized. If a client device is located more than a predetermineddistance from the garage (e.g., 30 meters, 50 meters, 100 meters, 500meters, 1 kilometer, 5 kilometers), the movement of the object may beclassified as unauthorized.

The time of day may also be incorporated into a determination of whetherthe movement of an object is authorized. For example, if an object ismoved during a predetermined time of day (e.g., when people are likelyto be asleep between 11 PM and 7 AM), the movement of the object may beclassified as unauthorized. A client may also set an away or vacationstatus, such that any movement of objects during that time is classifiedas unauthorized. Such a setting may be recurring or a one-time setting.For example, if a person is away at work each day from 8:30 AM to 6 PM,the system is configured by the client to classify any movement ofobjects in the garage during that time of day (on a recurring basis) tobe unauthorized. In another example, a client may input information thatthey will be on vacation for the next 7 days, and that any movement ofobjects in the garage for those 7 days (a one-time setting) should beclassified as unauthorized. A flexible recurring setting may also beused. For example, a client may set the system to monitor the garage andclassify movement as unauthorized while they are at work during the day.However, the client may not go to work and/or arrive home at exactly thesame time each day. Image data captured by the security camera 142(e.g., showing the client's vehicle leaving the garage) and/or locationdata associated with a client device and/or client vehicle device may beused to determine that a client has left the house in the morningregardless of the specific time at which the client left the house. Onceit is determined that the client has left the house, any movement in thegarage may be classified as unauthorized. Similarly, the location dataof the client device and/or client vehicle device and/or image datacaptured by the security camera 142 (e.g., showing the client's vehiclearriving back at the garage) may be used to stop considering anymovement of objects within the garage to be unauthorized. An examplemethod for monitoring objects in a garage is further described below andillustrated with respect to FIG. 12.

FIGS. 1D-1F are schematic diagrams of examples of a front view of agarage 152 with a garage door 162 in an open state, a partially openstate, and a closed state, respectively, according to various aspects ofthe present disclosure. A garage 152 is shown in FIGS. 1D-1F. In FIG.1D, the garage is open and a person 158 is standing in the opening ofthe garage 152. Mounted on the garage is a light camera 154. The lightcamera 154 may be used as described herein to capture image data thatmay be used at least in part to determine when the garage door 162should be opened and/or closed. In FIG. 1E, the garage door 162 is shownin an intermediate state, where the garage door 162 is not fully closedbut not fully open. In various embodiments, the garage door 162 as shownin FIG. 1E may be closing, opening, or may be suspended in theintermediate state shown.

The garage 152 also has a keypad 156. The keypad 156 is mounted on theoutside of the garage 152, such that the keypad 156 may be accessed by aperson outside of the garage 152 even if the garage door 162 is closed(e.g., as shown in FIG. 1F). The keypad 156 has buttons that a person,such as the person 158, may press to enter authorization codes to causethe garage door 162 to open, close, and/or stop moving. As describedherein, for example with respect to FIG. 19 below, the keypad 156 may beused for a single press open of a garage door. For example, the person158 may press a button on the keypad 156 when the garage door 162 isclosed (e.g., as shown in FIG. 1F). If the person is carrying a clientdevice that communicates with the keypad 156, the light camera 154, oranother device to authorize entry to the garage 152, the single buttonpress may cause the garage door 162 to open (e.g., as shown in FIG. 1D).The keypad 156 may also include a scanner such as an optical scanner.Such a scanner may be used to allow access to the garage, such as aguest access with a scannable code as described below with respect toFIG. 14. In various embodiments, the keypad 156 may also be used toenter a distress code, for example as described below with respect toFIG. 20. For example, if a person is under duress, that person may entera specific distress code into the keypad 156. The distress code causesthe garage door 162 to open (or close) normally so that anyone elsenearby or observing the person entering the codes behavior thinks thatnothing is happening other than the opening (or closing) of the garagedoor 162. However, the distress code may also trigger a transmission ofan alert to security monitoring services and/or authorities. In thisway, someone could alert security monitoring service and/or authoritiesto a problem without it appearing that they have actually reported aproblem.

In some embodiments, the keypad 156 may also be equipped with a camerasimilar to the A/V devices described herein. Similarly, a keypad may beincorporated into various A/V devices described herein. For example, acamera or other visual scanning system in the keypad 156 may be used toscan visual codes (e.g., a QR code) presented on a display of a clientdevice. These codes may be scanned and authorize to access a garage(e.g., have the garage door open).

A sensor 160 is also shown at the bottom of the opening of the garage152. The sensor 160 may be a photoelectric sensor that detects ifsomething is in the opening of the garage 152, or may be a motion sensorthat detects motion. The sensor 160 is used to determine if anything isin the way of the garage door 162 closing. The sensor 160 may also betied into a broader security system. For example, if a security systemis armed, the sensor 160 detecting motion triggers an alarm conditionfor the security system. The sensor 160 may also be used to determinethat image data should be captured by light camera 154. For example, ifa car is backing out of the garage 152, the sensor 160 will detectmotion, and the light camera 154 begins capturing image data to helpdetermine that the vehicle has fully left the garage 152. In anotherexample, the sensor 160 detecting motion may trigger a camera inside thegarage to begin capturing image data. Information from the sensor 160may further be used to determine when the garage door should be closed.For example, if something is in the opening (e.g., the person 158), thesensor 160 detects its presence and prevents the garage door 162 frombeing closed. Further examples of how the sensor 160 may be used aredescribed herein, for example with respect to FIG. 21 described below.

FIG. 2 is a functional block diagram illustrating a system 200 forcommunicating in a network according to various aspects of the presentdisclosure. Home automation, or smart home, is building automation forthe home. Home automation enable users (e.g., home owners and authorizedindividuals) to control and/or automate various devices and/or systems,such as lighting, heating (e.g., smart thermostats), ventilation, homeentertainment, air conditioning (HVAC), blinds/shades, security devices(e.g., contact sensors, smoke/CO detectors, motion sensors, etc.),washers/dryers, ovens, refrigerators/freezers, garage door openers,and/or other network connected devices suitable for use in the home. Invarious embodiments, Wi-Fi is used for remote monitoring and control ofsuch devices and/or systems. Smart home devices (e.g., hub devices 202,sensors 204, automation devices 206, a virtual assistant (VA) device208, Audio/Video (A/V) recording and communication devices 210, etc.),when remotely monitored and controlled via a network (Internet/a publicswitched telephone network (PSTN)) 212 (which may be similar to, andrepresent the network 112), may be considered to be components of the“Internet of Things.” Smart home systems may include switches and/orsensors (e.g., the sensors 204) connected to a central hub such as thesmart-home hub device 202 and/or the VA device 208 (the hub device 202and/or the VA device 208 may alternatively be referred to as a gateway,a controller, a home-automation hub, or an intelligent personalassistance device) from which the system may be controlled throughvarious user interfaces, such as voice commands and/or a touchscreen.Various examples, of user interfaces may include any or all of awall-mounted terminal (e.g., a keypad, a touchscreen, etc.), softwareinstalled on the client devices 214, 216, 230 (e.g., a mobileapplication), a tablet computer, or a web interface. Furthermore, theseuser interfaces are often but not always supported by Internet cloudservices. In one example, the Internet cloud services are responsiblefor obtaining user input via the user interfaces (e.g., a user interfaceof the hub device 202 and/or the VA device 208) and causing the smarthome devices (e.g., the sensors 204, the automation devices 206, etc.)to perform an operation in response to the user input. The keypad 156 ofFIGS. 1D-1F may include similar components and functionality as the userinterfaces described herein.

The hub device 202, the VA device 208, the sensors 204, the automationdevices 206, the A/V recording and communication devices 210, and/orclient devices 214, 216, 230 may use one or more wired and/or wirelesscommunication protocols to communicate, including, for example andwithout limitation, Wi-Fi (e.g., the user's network 218), X10, Ethernet,RS-485, 6LoWPAN, Bluetooth LE (BLE), ZigBee, Z-Wave, and/or a low powerwide-area networks (LPWAN), such as a chirp spread spectrum (CSS)modulation technology network (e.g., LoRaWAN), an Ultra Narrow Bandmodulation technology network (e.g., Sigfox, Telensa, NB-IoT, etc.),RingNet, and/or the like.

The user's network 218 may be, for example, a wired and/or wirelessnetwork. If the user's network 218 is wireless, or includes a wirelesscomponent, the user's network 218 may be a Wi-Fi network compatible withthe IEEE 802.11 standard and/or other wireless communicationstandard(s). Furthermore, the user's network 218 may be connected toother networks such as the network 212, which may comprise, for example,the Internet and/or PSTN. The devices, sensors, and actuator (e.g.,opener) of a garage door of FIGS. 1A-1F may communicate with similarcomponents and functionality as the user's network 218.

The system 200 may include one or more A/V recording and communicationdevices 210 (alternatively be referred to herein as “A/V devices 210” or“A/V device 210”) (which may represent, and/or be similar to, the A/Vrecording and communication devices 104, 118, 126, 142, and 154 of FIGS.1A-1F). The A/V devices 210 may include security cameras 210(a), lightcameras 210(b) (e.g., floodlight cameras, spotlight cameras, etc.),video doorbells 210(c) (e.g., wall powered and/or battery powered videodoorbells), and/or other devices capable of recording audio data and/orimage data. The A/V devices 210 may be configured to access a user'snetwork 218 to connect to a network (Internet/PSTN) 212 and/or may beconfigured to access a cellular network to connect to the network(Internet/PSTN) 212. The components and functionality of the A/V devices210 are described in more detail below with respect to FIG. 3. Thedevices, sensors, and actuator (e.g., opener) of a garage door of FIGS.1A-1F may communicate with similar components and functionality as thenetwork (Internet/PSTN) 212.

The system 200 may further include a smart-home hub device 202 (whichmay alternatively be referred to herein as the “hub device 202”)connected to the user's network 218 and/or the network (Internet/PSTN)212. The smart-home hub device 108 of FIG. 1A may include similarcomponents and functionality as the smart-home hub device 202 describedherein. The smart-home hub device 202 (also known as a home automationhub, gateway device, or network device), may comprise any device thatfacilitates communication with and control of the sensors 204,automation devices 206, the VA device 208, and/or the one or more A/Vdevices 210. For example, the smart-home hub device 202 may be acomponent of a security system and/or a home automation system installedat a location (e.g., a property, a premise, a home, a business, etc.).In some embodiments, the A/V devices 210, the VA device 208, the sensors204, and/or the automation devices 206 communicate with the smart-homehub device 202 directly and/or indirectly using one or more wirelessand/or wired communication protocols (e.g., BLE, Zigbee, Z-Wave, etc.),the user's network 218 (e.g., Wi-Fi, Ethernet, etc.), and/or the network(Internet/PSTN) 212. In some of the present embodiments, the A/V devices210, the VA device 208, the sensors 204, and/or the automation devices206 may, in addition to or in lieu of communicating with the smart-homehub device 202, communicate with the client devices 214, 216, 230, theVA device 208, and/or one or more of components of the network ofservers/backend devices 220 directly and/or indirectly via the user'snetwork 218 and/or the network (Internet/PSTN) 212.

As illustrated in FIG. 2, the system 200 includes the VA device 208. TheVA device 208 may be connected to the user's network 218 and/or thenetwork (Internet/PSTN) 212. The VA device 208 may include anintelligent personal assistant, such as, without limitation, AmazonAlexa® and/or Apple Siri® For example, the VA device 208 may beconfigured to receive voice commands, process the voice commands todetermine one or more actions and/or responses (e.g., transmit the voicecommands to the one or more components of the network of servers/backenddevices 220 for processing), and perform the one or more actions and/orresponses, such as to activate and/or change the status of one or moreof the sensors 204, automation devices 206, or A/V devices 210. In someembodiments, the VA device 208 is configured to process user inputs(e.g., voice commands) without transmitting information to the networkof servers/backend devices 220 for processing. The VA device 208 mayinclude at least one speaker (e.g., for playing music, for outputtingthe audio data generated by the A/V devices 210, for outputting thevoice of a digital assistant, etc.), at least one a microphone (e.g.,for receiving commands, for recording audio data, etc.), and a display(e.g., for displaying a user interface, for displaying the image datagenerated by the A/V devices 210, etc.). In various embodiments, the VAdevice 208 may include an array of speakers that are able to producebeams of sound. Although illustrated as a separate component in FIG. 2,in some embodiments the VA device 208 may not be a separate componentfrom the hub device 202. In such embodiments, the hub device 202 mayinclude the functionality of the VA device 208 or the VA device 208 mayinclude the functionality of the hub device 202.

The one or more sensors 204 may include, for example, at least one of adoor sensor, a window sensor, a contact sensor, a tilt sensor, atemperature sensor, a carbon monoxide sensor, a carbon dioxide sensor, aphotoelectric sensor, a smoke detector, a light sensor, a glass breaksensor, a freeze sensor, a flood sensor, a moisture sensor, a motionsensor, and/or other sensors that may provide the user/owner of thesecurity system a notification of a security event at his or herproperty. The sensor 160 of FIGS. 1D and 1E may include similarcomponents and functionality as the one or more sensors 204 describedherein.

In various embodiments, a contact sensor may include any componentconfigured to inform (e.g., via a signal) the security system whether anobject (e.g., a door or a window) is open or closed. A contact sensormay include first and second components: a first component installed onthe object itself (e.g., the door or the window); the second componentinstalled next to the object (e.g., on the door jamb). The first andsecond components of the contact sensor, however, need not actually bein physical contact with one another in order to be in the closed (notfaulted) state. For example, at least one of the first and secondcomponents may include a magnet, and the contact sensor may rely on theHall effect for determining a proximity of the first and second piecesto one another. When the door, window, or other object, is opened, andthe first and second components move apart from one another, the contactsensor may transmit an open signal to the security system (e.g., to thehub device 202). A similar process may be performed when the object isclosed. In some examples, a signal transmitted by the security system bythe contact sensor during opening and/or closing may be the same signal,and the hub device 202 may interpret the signal based on the known stateof the object (e.g., when a door is closed, and the signal is received,the hub device 202 may update the status of the door to open).

The one or more automation devices 206 may include, for example, atleast one of an outdoor lighting system, an indoor lighting system, andindoor/outdoor lighting system, a temperature control system (e.g., athermostat), a shade/blind control system, a locking control system(e.g., door lock, window lock, etc.), a home entertainment automationsystem (e.g., TV control, sound system control, etc.), an irrigationcontrol system, a wireless signal range extender (e.g., a Wi-Fi rangeextender, a Z-Wave range extender, etc.) a doorbell chime, a barriercontrol device (e.g., an automated door hinge), a smart doormat, agarage door opening/actuating system 206(a) and/or other automationdevices. The garage door actuators and opener systems described hereinthroughout may include similar components and functionality as thegarage door opening/actuating system 206(a) described herein.

As described herein, in some of the present embodiments, some or all ofthe client devices 214, 216, 230 the A/V device(s) 210, the smart-homehub device 202, the VA device 208, the sensors 204, and the automationdevices 206 may be referred to as a security system and/or ahome-automation system. The security system and/or home-automationsystem may be installed at location, such as a property, home, business,or premises for the purpose of securing and/or automating all or aportion of the location.

The system 200 may further include one or more client devices 214, 216,230 (which may represent, and/or be similar to, the client devices ofthe persons 110, 112, 124 of FIG. 1A and/or the client vehicle devicesof vehicles 122, 130, 136 of FIG. 1A). The client devices 214, 216, 230may communicate with and/or be associated with (e.g., capable of accessto and control of) the A/V devices 210, a smart-home hub device 202, theVA device 208, sensors 204, and/or automation devices 206. In variousembodiments, the client devices 214, 216, 230 communicate with otherdevices using one or more wireless and/or wired communication protocols,the user's network, and/or the network (Internet/PSTN) 212, as describedherein. The client devices 214, 216, 230 may comprise, for example, amobile device such as a smartphone or a personal digital assistant(PDA), or a computing device such as a tablet computer, a laptopcomputer, a desktop computer, etc. In some embodiments, the clientdevices 214, 216, 230 includes a connected device, such as a smartwatch, Bluetooth headphones, another wearable device, or the like. Insuch embodiments, the client devices 214, 216, 230 may include acombination of the smartphone or other device and a connected device(e.g., a wearable device), such that alerts, data, and/or informationreceived by the smartphone or other device are provided to the connecteddevice, and one or more controls of the smartphone or other device maybe input using the connected device (e.g., by touch, voice, etc.).

The A/V devices 210, the hub device 202, the VA device 208, theautomation devices 206, the sensors 204, and/or the client devices 214,216, 230 may also communicate, via the user's network 218 and/or thenetwork (Internet/PSTN) 212, with network(s) of servers and/or backenddevices 220, such as (but not limited to) one or more remote storagedevices 222 (may be referred to interchangeably as “cloud storagedevice(s)”), one or more backend servers 224, and one or more backendapplication programming interfaces (APIs) 226. While FIG. 2 illustratesthe storage device 222, the backend server 224, and the backend API 226as components separate from the network 220, it is to be understood thatthe storage device 222, the backend server 224, and/or the backend API226 may be considered to be components of the network 220. For example,the network 220 may include a data center with a plurality of computingresources used to implement the storage device 222, the backend server224, and the backend API 226. The devices, sensors, and actuator (e.g.,opener) of a garage door of FIGS. 1A-1F may communicate with one anothervia similar components and functionality as the network 220 and maycommunicate with the various components of FIG. 2 via the network 220 asdescribed herein.

The backend server 224 may comprise a computer program or other computerexecutable code that, when executed by processor(s) of the backendserver 224, causes the backend server 224 to wait for requests fromother computer systems or software (clients) and provide responses. Inan embodiment, the backend server 224 shares data and/or hardware and/orsoftware resources among the client devices 214, 216, 230. Thisarchitecture is called the client-server model. The client devices 214,216, 230 may run on the same computer or may connect to the backendserver 224 over the network (Internet/PSTN) 212 and/or the network 220.Examples of computing servers include database servers, file servers,mail servers, print servers, web servers, game servers, and applicationservers. The term server may be construed broadly to include anycomputerized process that shares a resource to one or more clientprocesses.

The backend API 226 may comprise, for example, a server (e.g. a realserver, or a virtual machine, or a machine running in a cloudinfrastructure as a service), or multiple servers networked together,exposing at least one API to clients. In various embodiments, thebackend API 226 is provided by servers including various components suchas an application server (e.g. software servers), a caching layer, adatabase layer, or other components suitable for implementing one ormore APIs. The backend API 226 may, for example, comprise a plurality ofapplications, each of which communicate with one another using one ormore public APIs. In some embodiments, the backend API 226 maintainsuser data and provides user management capabilities, thereby reducingthe load (e.g., memory and processor consumption) of the client devices214, 216, 230.

In various embodiments, an API is a set of routines, protocols, andtools for building software and applications. Furthermore, the API maydescribe a software component in terms of its operations, inputs,outputs, and underlying types, defining functionalities that areindependent of their respective implementations, which allowsdefinitions and implementations to vary without compromising theinterface. As such, the API may provide a programmer with access to aparticular application's functionality without the need to modify theparticular application.

The backend API 226 illustrated in FIG. 2 may further include one ormore services (also referred to as network services). A network serviceis an application that provides data storage, manipulation,presentation, communication, and/or other capability. Network servicesare often implemented using a client-server architecture based onapplication-layer network protocols. Each service may be provided by aserver component (e.g., the backend server 224) running on one or morecomputers (such as a dedicated server computer offering multipleservices) and accessed via a network by client components running onother devices (e.g., client devices 214, 216, 230). However, the clientand server components can both be run on the same machine. Clients andservers may have a user interface, and sometimes other hardwareassociated with them.

The network 220 may be any wireless network, any wired network, or acombination thereof, configured to operatively couple theabove-mentioned modules, devices, components, and/or systems asillustrated in FIG. 2. For example, the network 220, the user's network218, and/or the network (Internet PSTN) 212 may include one or more ofthe following: a PSTN (public switched telephone network), the Internet,a local intranet, a PAN (Personal Area Network), a LAN (Local AreaNetwork), a WAN (Wide Area Network), a MAN (Metropolitan Area Network),a virtual private network (VPN), a storage area network (SAN), a framerelay connection, an Advanced Intelligent Network (AIN) connection, asynchronous optical network (SONET) connection, a digital T1, T3, E1 orE3 line, a Digital Data Service (DDS) connection, a DSL (DigitalSubscriber Line) connection, an Ethernet connection, an ISDN (IntegratedServices Digital Network) line, a dial-up port such as a V.90, V.34, orV.34bis analog modem connection, a cable modem, an ATM (AsynchronousTransfer Mode) connection, or an FDDI (Fiber Distributed Data Interface)or CDDI (Copper Distributed Data Interface) connection. Furthermore,communications may also include links to any of a variety of wirelessnetworks, including WAP (Wireless Application Protocol), GPRS (GeneralPacket Radio Service), GSM (Global System for Mobile Communication),LTE, VoLTE, LoRaWAN, LPWAN, RPMA, LTE Cat-“X” (e.g. LTE Cat 1, LTE Cat0, LTE CatM1, LTE Cat NB1), CDMA (Code Division Multiple Access), TDMA(Time Division Multiple Access), FDMA (Frequency Division MultipleAccess), and/or OFDMA (Orthogonal Frequency Division Multiple Access)cellular phone networks, global navigation satellite system (GNSS), suchas global positioning systems (GPS), CDPD (cellular digital packetdata), RIM (Research in Motion, Limited) duplex paging network,Bluetooth radio, or an IEEE 802.11-based radio frequency network. Thenetwork can further include or interface with any one or more of thefollowing: RS-232 serial connection, IEEE-4024 (Firewire) connection,Fibre Channel connection, IrDA (infrared) port, SCSI (Small ComputerSystems Interface) connection, USB (Universal Serial Bus) connection, orother wired or wireless, digital or analog, interface or connection,mesh or Digi® networking.

The hub device 202, the VA device 208, and/or any of the components ofthe network(s) of servers/backend devices 220 (e.g., the backend server224, the backend API 226, the storage devices 222, etc.) may be referredto herein as a “network device” or “network devices.” The devices,sensors, and actuator (e.g., opener) of a garage door of FIGS. 1A-1F maycommunicate with one another and/or the components of FIG. 2 via similarcomponents and functionality as the network devices described herein.

With further reference to FIG. 2, the system 200 may also include asecurity monitoring service 228. The security monitoring service 228 maybe operated by the same company that manufactures, sells, and/ordistributes the A/V devices 210, the hub device 202, the VA device 208,the sensors 204, and/or the automation devices 206. In otherembodiments, the security monitoring service 228 may be operated by athird-party company (e.g., a different company than the one thatmanufactured, sold, and/or distributed the A/V devices 210, the hubdevice 202, the VA device 208, the sensors 204, and/or the automationdevices 206). In any of the present embodiments, the security monitoringservice 228 may have control of at least some of the features andcomponents of the security system and/or the home-automation system(e.g., the security monitoring service 228 may be able to arm and/ordisarm the security system, lock and/or unlock doors, activate and/ordeactivate one or more of the sensors 204 and/or the automation devices206, etc.). For example, the security monitoring service 228 may operateand control their own client devices and/or network of servers/backenddevices for monitoring and/or controlling security systems. In such anexample, the A/V devices 210, the hub device 202, the VA device 208, thesensors 204, and/or the automation devices 206 may communicate with theclient devices and/or one or more components of the network ofservers/backend devices of the security monitoring service 228 over thenetwork (Internet/PSTN) 212 (in some embodiments, via one or more of thecomponents of the network of backend servers/backend devices 220).

FIG. 3 is a functional block diagram for an audio/video (A/V) deviceaccording to various aspects of the present disclosure. In someembodiments, the one or more A/V devices 210 may include the securitycamera 210(a). In other embodiments, the one or more A/V devices 210 mayinclude the light camera 210(b), which may include some or all of thecomponents of the security camera 210(a) in addition to a lightcontroller 302 and one or more lights 304(a), 304(b). In someembodiments, the one or more A/V devices 210 may include the videodoorbell 210(c), which may include some or all of the components of thesecurity camera 210(a) in addition to a button 306, and in someembodiments, a connection to a signaling device 308 (e.g., apre-installed signaling device, such as a wired signaling device, and/ora wireless signaling device, connected over Wi-Fi, BLE, or anotherwireless communication protocol). The video doorbell 104, the lightcameras 126, 154, and the security cameras 118, 142 of FIGS. 1A-1F mayinclude similar components and functionality as the A/V devices 210described herein.

With further reference to FIG. 3, the A/V device 210 may include aprocessor(s) 310, a communication module 312, a camera 314, a computervision module 316, a light sensor 318, an audio CODEC (coder-decoder)320, volatile memory 322, and non-volatile memory 324. The processor(s)310 (alternatively referred to herein as a “CPU,” a “controller,” and/ora “microcontroller) may comprise an integrated circuit including aprocessor core, memory, and programmable input/output peripherals. Theprocessor(s) 310 may receive input signals, such as data and/or power,from the camera 314, motion sensor(s) 326, light sensor 318,microphone(s) 328, speaker(s) 330, and/or the communication module 312,and may perform various functions as described in the presentdisclosure. In various embodiments, when the processor(s) 310 istriggered by the motion sensor(s) 326, the camera 314, the speaker(s)330, the microphone(s) 328, the communication module 312, and/or anothercomponent, the processor(s) 310 performs one or more processes and/orfunctions. For example, when the light sensor 318 detects a low level ofambient light, the light sensor 318 may trigger the processor(s) 310 toenable a night vision camera mode. The processor(s) 310 may also providedata communication between various components such as between thecommunication module 312 and the camera 314.

With further reference to FIG. 3, the communication module 312 maycomprise an integrated circuit including a processor core, memory, andprogrammable input/output peripherals. The communication module 312 maybe operatively connected to the processor(s) 310. In some embodiments,the communication module 312 is configured to handle communication linksbetween the A/V device 210 and other, external devices, externalreceivers, external transmitters, and/or external transceivers, and toroute incoming/outgoing data appropriately. For example, inbound datafrom an antenna 332 of the communication module 312 may be routedthrough the communication module 312 before being directed to theprocessor(s) 310, and outbound data from the processor(s) 310 may berouted through the communication module 312 before being directed to theantenna 332 of the communication module 312. As another example, thecommunication module 312 may be configured to transmit data to and/orreceive data from a remote network device (e.g., one or more componentsof the network(s) of servers/backend devices 220 described in FIG. 2).The communication module 312 may include wireless 334(a) and wired334(b) adapters. For example, the communication module 312 may includeone or more wireless antennas, radios, receivers, transmitters, and/ortransceivers (not shown in FIG. 3 for simplicity) configured to enablecommunication across one or more wireless networks, such as, withoutlimitation, Wi-Fi, cellular, Bluetooth, Z-Wave, Zigbee, LPWAN(s), and/orsatellite networks. The communication module 312 may receive inputs,such as power and/or data, from the camera 314, the processor(s) 310,the button 306 (in embodiments where the A/V device 210 is the videodoorbell 210(c)), the motion sensors 326, a reset button (not shown inFIG. 3 for simplicity), and/or the non-volatile memory 324. Thecommunication module 312 may also include the capability ofcommunicating over wired connections, such as with a signaling device308. For example, when the button 306 of the video doorbell 210(c) ispressed, the communication module 312 may be triggered to perform one ormore functions, such as to transmit a signal over the wired 334(b)connection to the signaling device 308 (although, in some embodiments,the signal be transmitted over a wireless 334(a) connection to thesignaling device) to cause the signaling device 308 to emit a sound(e.g., a doorbell tone, a user customized sound, a ringtone, a seasonalringtone, etc.). The communication module 312 may also act as a conduitfor data communicated between various components and the processor(s)310.

With further reference to FIG. 3, the A/V device 210 may include thenon-volatile memory 324 and the volatile memory 322. The non-volatilememory 324 may comprise flash memory configured to store and/or transmitdata. For example, in certain embodiments the non-volatile memory 324may comprise serial peripheral interface (SPI) flash memory. In someembodiments, the non-volatile memory 324 may comprise, for example, NANDor NOR flash memory. The volatile memory 322 may comprise, for example,DDR3 SDRAM (double data rate type three synchronous dynamicrandom-access memory). In the embodiment illustrated in FIG. 3, thevolatile memory 322 and the non-volatile memory 324 are illustrated asbeing separate from the processor(s) 310. However, the illustration ofFIG. 3 is not intended to be limiting, and in some embodiments thevolatile memory 322 and/or the non-volatile memory 324 may be physicallyincorporated with the processor(s) 310, such as on the same chip. Thevolatile memory 322 and/or the non-volatile memory 324, regardless oftheir physical location, may be shared by one or more other components(in addition to the processor(s) 310) of the present A/V device 210.

With further reference to FIG. 3, the A/V device 210 may include thecamera 314. The camera 314 may include an image sensor 336. The imagesensor 336 may include a video recording sensor and/or a camera chip. Inone aspect of the present disclosure, the image sensor 336 may comprisea complementary metal-oxide semiconductor (CMOS) array and may becapable of recording high definition (e.g., 722p, 1800p, 4K, etc.) videofiles. The camera 314 may include a separate camera processor (not shownin FIG. 3 for simplicity), or the processor(s) 310 may perform thecamera processing functionality. The processor(s) 310 (and/or cameraprocessor) may include an encoding and compression chip. In someembodiments, the processor(s) 310 (and/or the camera processor) maycomprise a bridge processor. The processor(s) 310 (and/or the cameraprocessor) may process video recorded by the image sensor 336 and/oraudio recorded by the microphone(s) 328, and may transform this datainto a form suitable for transfer by the communication module 312 to thenetwork (Internet/PSTN) 212. In various embodiments, the camera 314 alsoincludes memory, such as volatile memory that may be used when data isbeing buffered or encoded by the processor(s) 310 (and/or the cameraprocessor). For example, in certain embodiments the camera memory maycomprise synchronous dynamic random-access memory (SD RAM).

The camera 314 may further include an IR cut filter 338 that maycomprise a system that, when triggered, configures the image sensor 336to see primarily infrared light as opposed to visible light. Forexample, when the light sensor 318 detects a low level of ambient light(which may comprise a level that impedes the performance of the imagesensor 336 in the visible spectrum), the light emitting components 229may shine infrared light through an enclosure of the A/V device 210 outto the environment, and the IR cut filter 338 may enable the imagesensor 336 to see this infrared light as it is reflected or refractedoff of objects within the field of view of the doorbell. This processmay provide the A/V device with the “night vision” function mentionedabove.

With further reference to FIG. 3, the recording and communication A/Vdevice 210 may comprise the light sensor 318 and the one or morelight-emitting components 340, such as LED's. The light sensor 318 maybe one or more sensors capable of detecting the level of ambient lightof the surrounding environment in which the A/V device 210 may belocated. The light-emitting components 340 may be one or morelight-emitting diodes capable of producing visible light when suppliedwith power (e.g., to enable night vision). In some embodiments, whenactivated, the light-emitting components 340 illuminates a light pipe.

The A/V device 210 may further include one or more speaker(s) 330 and/orone or more microphone(s) 328. The speaker(s) 330 may be anyelectromechanical device capable of producing sound in response to anelectrical signal input. The microphone(s) 328 may be anacoustic-to-electric transducer or sensor capable of converting soundwaves into an electrical signal. In some embodiments, the A/V device 210may include two or more microphone(s) 328 that are spaced from oneanother (e.g., located on different sides of the A/V device 210) toprovide noise cancelling and/or echo cancelling for clearer audio. Thespeaker(s) 330 and/or microphone(s) 328 may be coupled to an audio CODEC320 to enable digital audio received by client devices to bedecompressed and output by the speaker(s) 330 and/or to enable audiodata captured by the microphone(s) 328 to be compressed into digitalaudio data. The digital audio data may be received from and transmittedto client devices using the communication module 312 (in someembodiments, through one or more intermediary devices such as the hubdevice 202, the VA device 208, and/or one or more components of thenetwork of servers/backend devices 220 as described in FIG. 2). Forexample, when a visitor (or intruder) who is present in the area aboutthe A/V device 210 speaks, sound from the visitor (or intruder) isreceived by the microphone(s) 328 and compressed by the audio CODEC 320.Digital audio data is then sent through the communication module 312 tothe network 212 via the user's network 218, routed by the backend server224 and/or the backend API 226 and delivered to the client device(s)214, 216, 230 as described above in connection with FIG. 2. When theuser speaks, after being transferred through the network 212, the user'snetwork 218, and the communication module 312, the digital audio datafrom the user is decompressed by the audio CODEC 320 and emitted to thevisitor through the speaker(s) 330.

With further reference to FIG. 3, the A/V device 210 may be batterypowered using a battery 342 and/or may be powered using a source ofexternal AC (alternating-current) power, such as a household AC powersupply (alternatively referred to herein as “AC mains” or “wall power”).The AC power may have a voltage in the range of 110-220 VAC, forexample. The incoming AC power may be received by an AC/DC adapter (notshown), which may convert the incoming AC power to DC (direct-current)and may step down the voltage from 110-220 VAC to a lower output voltageof about 12 VDC and an output current of about 2 A, for example. Invarious embodiments, the output of the AC/DC adapter is in a range fromabout 9 V to about 15 V and in a range from about 0.5 A to about 5 A.These voltages and currents are examples provided for illustration andare not intended to be limiting.

However, in other embodiments, a battery 342 may not be included. Inembodiments that include the battery 342, the A/V device 210 may includean integrated circuit (not shown) capable of arbitrating betweenmultiple voltage rails, thereby selecting the source of power for theA/V device 210. The A/V device 210 may have separate power railsdedicated to the battery 342 and the AC power source. In one aspect ofthe present disclosure, the A/V device 210 may continuously draw powerfrom the battery 342 to power the A/V device 210, while at the same timerouting the AC power to the battery, thereby allowing the battery 342 tomaintain a substantially constant level of charge. Alternatively, theA/V device 210 may continuously draw power from the AC power to powerthe doorbell, while only drawing from the battery 342 when the AC poweris low or insufficient. Still, in some embodiments, the battery 342comprises the sole source of power for the A/V device 210. In suchembodiments, the components of the A/V device 210 (e.g., springcontacts, connectors, etc.) are not be connected to a source of ACpower. When the battery 342 is depleted of its charge, it may berecharged, such as by connecting a power source to the battery 342(e.g., using a USB connector).

Although not illustrated in FIG. 3, in some embodiments, the A/V device210 may include one or more of an accelerometer, a barometer, a humiditysensor, and a temperature sensor. The accelerometer may be one or moresensors capable of sensing motion and/or acceleration. The one or moreof the accelerometer, the barometer, the humidity sensor, and thetemperature sensor may be located outside of a housing of the A/V device210 so as to reduce interference from heat, pressure, moisture, and/orother stimuli generated by the internal components of the A/V device210. The barometer, the humidity sensor, and the temperature sensor maymeasure atmospheric conditions that yield atmospheric data. Such sensorsmay be used, for example, in a method such as the method described belowwith respect to FIG. 23.

With further reference to FIG. 3, the A/V device 210 may include one ormore motion sensor(s) 326. However, in some embodiments, the motionsensor(s) 326 may not be included, such as where motion detection isperformed by the camera 314 or another device. The motion sensor(s) 326may be any type of sensor capable of detecting and communicating thepresence of an entity within their field of view. As such, the motionsensor(s) 326 may include one or more (alone or in combination)different types of motion sensors. For example, in some embodiments, themotion sensor(s) 326 may comprise passive infrared (PIR) sensors, whichmay be secured on or within a PIR sensor holder that may reside behind alens (e.g., a Fresnel lens). In such an example, the PIR sensors maydetect IR radiation in a field of view, and produce an output signal(typically a voltage) that changes as the amount of IR radiation in thefield of view changes. The amount of voltage in the output signal may becompared, by the processor(s) 310, for example, to one or more thresholdvoltage values to determine if the amount of voltage in the outputsignal is indicative of motion, and/or if the amount of voltage in theoutput signal is indicative of motion of an entity that is to becaptured by the camera 314 (e.g., motion of a person and/or animal mayprompt activation of the camera 314, while motion of a vehicle may not).In some embodiments, motion of an entity that is to be captured by thecamera 314 may be a vehicle so as to implement the various methods andprocesses described herein. Although the above discussion of the motionsensor(s) 326 primarily relates to PIR sensors, depending on theembodiment, the motion sensor(s) 326 may include additional and/oralternate sensor types that produce output signals including alternativedata types. For example, and without limitation, the output signal mayinclude an amount of voltage change based on the presence of infraredradiation in a field of view of an active infrared (AIR) sensor, theoutput signal may include phase shift data from a microwave-type motionsensor, the output signal may include doppler shift data from anultrasonic-type motion sensor, the output signal may include radio wavedisturbance from a tomographic-type motion sensor, and/or the outputsignal may include other data types for other sensor types that may beused as the motion sensor(s) 326 of the A/V device 210.

In some embodiments, computer vision module(s) (CVM) 316 may be includedin the A/V device 210 as the motion sensor(s) 326, in addition to, oralternatively from, other motion sensor(s) 326. For example, the CVM 316may be a low-power CVM (e.g., Qualcomm Glance) that, by operating at lowpower (e.g., less than 2 mW of end-to-end power), is capable ofproviding computer vision capabilities and functionality for batterypowered devices (e.g., the A/V device 210 when powered by the battery342). The low-power CVM may include a lens, a CMOS image sensor, and adigital processor that may perform embedded processing within thelow-power CVM itself, such that the low-power CVM may outputpost-processed computer vision metadata to the processor(s) 310 (e.g.,via a serial peripheral bus interface (SPI)). As such, the low-power CVMmay be considered to be one or more of the motion sensor(s) 326, and thedata type output in the output signal may be the post-processed computervision metadata. The metadata may include information such as thepresence of a particular type of entity (e.g., person, animal, vehicle,parcel, etc.), a direction of movement of the entity, a distance of theentity from the A/V device 210, etc. This metadata may be used, forexample, to track an inventory of a garage or create an access log ofwho/what has entered or exited a garage as described herein (e.g., withrespect to FIGS. 1B, 1C, 12). The metadata may also be used to open orclose a garage door as described herein (e.g., with respect to FIGS. 8,10, 11, 13-19). In various embodiments, the motion sensor(s) 326 includea plurality of different sensor types capable of detecting motion suchas PIR, AIR, low-power CVM, and/or cameras.

As indicated above, the A/V device 210 may include the CVM 316 (whichmay be the same as the above described low-power CVM 316 implemented asone or more motion sensor(s) 326, or may be additional to, oralternative from, the above described low-power CVM 316). For example,the A/V device 210, the hub device 202, the VA device 208, and/or one ormore component of the network(s) of servers/backend devices 220 mayperform any or all of the computer vision processes and functionalitiesdescribed herein. In addition, although the CVM 316 is only illustratedas a component of the A/V device 210, the computer vision module 316 mayadditionally, or alternatively, be included as a component of the hubdevice 202, the VA device 208, and/or one or more components of thenetwork of servers/backend devices 220. With respect to the A/V device210, the CVM 316 may include any of the components (e.g., hardware)and/or functionality described herein with respect to computer vision,including, without limitation, one or more cameras, sensors, and/orprocessors. In some of the present embodiments, with reference to FIG.3, the microphone(s) 328, the camera 314, the processor(s) 310, and/orthe image sensor 336 may be components of the CVM 316. In someembodiments, the CVM 316 may include an internal camera, image sensor,and/or processor, and the CVM 316 may output data to the processor(s)310 in an output signal, for example.

As a result of including the CVM 316, some of the present embodimentsmay leverage the CVM 316 to implement computer vision for one or moreaspects, such as motion detection, object recognition, and/or facialrecognition. Computer vision includes methods for acquiring, processing,analyzing, and understanding images and, in general, high-dimensionaldata from the real world in order to produce numerical or symbolicinformation, e.g., in the form of decisions. Computer vision seeks toduplicate the abilities of human vision by electronically perceiving andunderstanding an image. Understanding in this context means thetransformation of visual images (the input of the retina) intodescriptions of the world that can interface with other thoughtprocesses and elicit appropriate action. This image understanding can beseen as the disentangling of symbolic information from image data usingmodels constructed with the aid of geometry, physics, statistics, andlearning theory. Computer vision has also been described as theenterprise of automating and integrating a wide range of processes andrepresentations for vision perception. As a scientific discipline,computer vision is concerned with the theory behind artificial systemsthat extract information from images. The image data can take manyforms, such as video sequences, views from multiple cameras, ormulti-dimensional data from a scanner.

One aspect of computer vision comprises determining whether or not theimage data contains some specific object, feature, or activity.Different varieties of computer vision recognition include: ObjectRecognition (also called object classification)—One or severalpre-specified or learned objects or object classes can be recognized,usually together with their 2D positions in the image or 3D poses in thescene. Identification—An individual instance of an object is recognized.Examples include identification of a specific person's face orfingerprint, identification of handwritten digits, or identification ofa specific vehicle. Detection—The image data are scanned for a specificcondition. Examples include detection of possible abnormal cells ortissues in medical images or detection of a vehicle in an automatic roadtoll system. Detection based on relatively simple and fast computationsis sometimes used for finding smaller regions of interesting image datathat can be further analyzed by more computationally demandingtechniques to produce a correct interpretation. Detection of vehicles,objects, persons, etc. can be used to implement the various systems,methods, and processes described herein throughout.

Several specialized tasks based on computer vision recognition exist,such as: Optical Character Recognition (OCR)—Identifying characters inimages of printed or handwritten text, usually with a view to encodingthe text in a format more amenable to editing or indexing (e.g., ASCII).OCR may be used, for example, for determining characters on a licenseplate (e.g., license plate number, state of licensure, expiration oflicense) or on a vehicle (e.g., the make, model, and/or trim package ofa vehicle). 2D Code Reading—Reading of 2D codes such as data matrix andQR codes. Facial Recognition. Shape Recognition Technology(SRT)—Differentiating human beings (e.g., head and shoulder patterns)from objects and/or vehicles.

Image acquisition—A digital image is produced by one or several imagesensors, which, besides various types of light-sensitive cameras, mayinclude range sensors, tomography devices, radar, ultra-sonic cameras,etc. Depending on the type of sensor, the resulting image data may be a2D image, a 3D volume, or an image sequence. The pixel values maycorrespond to light intensity in one or several spectral bands (grayimages or color images), but can also be related to various physicalmeasures, such as depth, absorption or reflectance of sonic orelectromagnetic waves, or nuclear magnetic resonance. For example,physical measures (e.g., depth information) may be used according tovarious methods and processes described herein. For example, physicalmeasures (e.g., depth information) may be used to determine vehiclesthat are approaching, leaving, entering, exiting, or otherwise moving inor around a garage and/or garage door as described herein throughout(e.g., with respect to FIGS. 1A-1F, 8-12, and/or 22). Physical measures(e.g., depth information) may also be used to keep track of objects in agarage, such as described herein with respect to FIGS. 1B, 1C, and/or12. Physical measures (e.g., depth information) may also be used to giveinstructions for placing an object or parking a vehicle within a garage,such as described herein with respect to FIGS. 1A-1C and 9. Physicalmeasures (e.g., depth information) may also be used to prevent a garagedoor from closing as a person moves out of a garage, such as describedherein with respect to FIG. 15.

Pre-processing—Before a computer vision method can be applied to imagedata in order to extract some specific piece of information, it isusually beneficial to process the data in order to assure that itsatisfies certain assumptions implied by the method. Examples ofpre-processing include, but are not limited to re-sampling in order toassure that the image coordinate system is correct, noise reduction inorder to assure that sensor noise does not introduce false information,contrast enhancement to assure that relevant information can bedetected, and scale space representation to enhance image structures atlocally appropriate scales.

Feature extraction—Image features at various levels of complexity areextracted from the image data. Typical examples of such features are:Lines, edges, and ridges; Localized interest points such as corners,blobs, or points; More complex features may be related to texture,shape, or motion.

Detection/segmentation—At some point in the processing a decision may bemade about which image points or regions of the image are relevant forfurther processing. Examples are: Selection of a specific set ofinterest points; Segmentation of one or multiple image regions thatcontain a specific object of interest; Segmentation of the image intonested scene architecture comprising foreground, object groups, singleobjects, or salient object parts (also referred to as spatial-taxonscene hierarchy).

High-level processing—At this step, the input may be a small set ofdata, for example a set of points or an image region that is assumed tocontain a specific object. The remaining processing may comprise, forexample: Verification that the data satisfy model-based andapplication-specific assumptions; Estimation of application-specificparameters, such as object pose or object size; Imagerecognition—classifying a detected object into different categories;Image registration—comparing and combining two different views of thesame object.

Decision making—Making the final decision required for the application,for example match/no-match in recognition applications.

One or more of the present embodiments may include a vision processingunit (not shown separately, but may be a component of the CVM 316). Avision processing unit is an emerging class of microprocessor; it is aspecific type of AI (artificial intelligence) accelerator designed toaccelerate machine vision tasks. Vision processing units are distinctfrom video processing units (which are specialized for video encodingand decoding) in their suitability for running machine vision algorithmssuch as convolutional neural networks, SIFT, etc. Vision processingunits may include direct interfaces to take data from cameras (bypassingany off-chip buffers), and may have a greater emphasis on on-chipdataflow between many parallel execution units with scratchpad memory,like a manycore DSP (digital signal processor). But, like videoprocessing units, vision processing units may have a focus on lowprecision fixed-point arithmetic for image processing.

Some of the present embodiments may use facial recognition hardwareand/or software, as a part of the computer vision system. Various typesof facial recognition exist, some or all of which may be used in thepresent embodiments.

Some face recognition algorithms identify facial features by extractinglandmarks, or features, from an image of the subject's face. Forexample, an algorithm may analyze the relative position, size, and/orshape of the eyes, nose, cheekbones, and jaw. These features are thenused to search for other images with matching features. Other algorithmsnormalize a gallery of face images and then compress the face data, onlysaving the data in the image that is useful for face recognition. Aprobe image is then compared with the face data. One of the earliestsuccessful systems is based on template matching techniques applied to aset of salient facial features, providing a sort of compressed facerepresentation.

Recognition algorithms can be divided into two main approaches,geometric, which looks at distinguishing features, or photometric, whichis a statistical approach that distills an image into values andcompares the values with templates to eliminate variances.

Popular recognition algorithms include principal component analysisusing eigenfaces, linear discriminant analysis, elastic bunch graphmatching using the Fisherface algorithm, the hidden Markov model, themultilinear subspace learning using tensor representation, and theneuronal motivated dynamic link matching.

Further, a newly emerging trend, claimed to achieve improved accuracy,is three-dimensional face recognition. This technique uses 3D sensors tocapture information about the shape of a face. This information is thenused to identify distinctive features on the surface of a face, such asthe contour of the eye sockets, nose, and chin.

One advantage of 3D face recognition is that it is not affected bychanges in lighting like other techniques. It can also identify a facefrom a range of viewing angles, including a profile view.Three-dimensional data points from a face vastly improve the precisionof face recognition. 3D research is enhanced by the development ofsophisticated sensors that do a better job of capturing 3D face imagery.The sensors work by projecting structured light onto the face. Up to adozen or more of these image sensors can be placed on the same CMOSchip—each sensor captures a different part of the spectrum.

Another variation is to capture a 3D picture by using three trackingcameras that point at different angles; one camera pointing at the frontof the subject, a second one to the side, and a third one at an angle.All these cameras work together to track a subject's face in real timeand be able to face detect and recognize.

Another emerging trend uses the visual details of the skin, as capturedin standard digital or scanned images. This technique, called skintexture analysis, turns the unique lines, patterns, and spots apparentin a person's skin into a mathematical space.

Another form of taking input data for face recognition is by usingthermal cameras, which may only detect the shape of the head and ignorethe subject accessories such as glasses, hats, or make up.

Further examples of automatic identification and data capture (AIDC)and/or computer vision that can be used in the present embodiments toverify the identity and/or authorization of a person include, withoutlimitation, biometrics. Biometrics refers to metrics related to humancharacteristics. Biometrics authentication (or realistic authentication)is used in various forms of identification and access control. Biometricidentifiers are the distinctive, measurable characteristics used tolabel and describe individuals. Biometric identifiers can bephysiological characteristics and/or behavioral characteristics.Physiological characteristics may be related to the shape of the body.Examples include, but are not limited to, fingerprints, palm veins,facial recognition, three-dimensional facial recognition, skin textureanalysis, DNA, palm prints, hand geometry, iris recognition, retinarecognition, and odor/scent recognition. Behavioral characteristics maybe related to the pattern of behavior of a person, including, but notlimited to, typing rhythm, gait, and voice recognition.

The present embodiments may use any one, or any combination of more thanone, of the foregoing biometrics to identify and/or authenticate aperson who is either suspicious or who is authorized to take certainactions with respect to a property or expensive item of collateral. Forexample, with reference to FIG. 3, the CVM 316, and/or the camera 314and/or the processor(s) 310 may receive information about the personusing any one, or any combination of more than one, of the foregoingbiometrics. Similarly, the present embodiments may use any one, or acombination of more than one, of the foregoing recognition methodsand/or algorithms to identify and/or authenticate a vehicle that issuspicious or that is authorized to access a property and/or a garage asdescribed herein.

Again, with reference to FIG. 3, in embodiments where the A/V device 210includes a light camera, the A/V device 210 may include the lightcontroller 302 and one or more lights 304(a), 304(b) (collectivelyreferred to herein as “lights 304”). The light controller 302 mayinclude a switch for controlling the lights 304. For example, inresponse to the motions sensor(s) 326 and/or the camera 314 detectingmotion, the light controller 236 may receive an output signal from theprocessor(s) 310 that causes the light controller 302 to activate theone or more lights 304(a), 304(b). In some embodiments, the light cameramay include motion sensor(s) 326 detecting motion for controllingactivation of the lights 304, and may further include the camera 314 fordetecting motion for activating the recording of the image data usingthe camera 314 and/or the recording of the audio data using themicrophone(s) 328. In other embodiments, the motion sensor(s) 326 maydetect the motion for activating the lights 304, the camera 314, and themicrophone(s) 328, or the camera 314 may detect the motion foractivating the lights 304, the camera 314 to being recording the imagedata, and the microphone(s) 328 to being recording the audio data. Thelights 304 may include floodlights, spotlights, porch lights, or anothertype of illumination device. The lights 304 may provide for better imagedata quality when ambient light levels are low (e.g., at dusk, dawn, ornight), while also providing a deterrent effect by being illuminatedwhen motion is detected.

With further reference to FIG. 3, in embodiments where the A/V device210 includes a doorbell, such as the video doorbell 210(c), the A/Vdevice 210 may include the button 306. In embodiments where the button306 is a mechanical button (e.g., has a range of movement), the button306 may make contact with a button actuator located within the videodoorbell 210(c) when the button 306 is pressed. In embodiments where thebutton 306 is not mechanical (e.g., has no range of motion), the button306 may include a capacitive touch button, a resistive touch button, asurface acoustic wave (SAW) button, an infrared (IR) button, an opticalimaging button, an acoustic pulse recognition button, and/or a buttonthat implements a low-power CVM for the detection of a person (e.g., afinger, hand, etc., of a person). When the button 306 is pressed,touched, and/or otherwise triggered, the processor(s) 310 may receive anoutput signal from the button 306 that may activate one or morefunctions of the video doorbell 210(c), such as transmitting an outputsignal, using the communication module 312, to the signaling device 308to cause the signaling device 308 to output a sound (e.g., via the wired334(b) connection to the signaling device 308 and/or a wireless 334(a)connection to the signaling device 308). In addition, the processor(s)310 may transmit an output signal (e.g., a message), using thecommunication module 312, to the client device(s) 214, 216, 230 toindicate to the user(s) of the client device(s) 214, 216, 230 that aperson is present at the A/V device 210 (in some embodiments, via atleast one of the hub device 202, the VA device 208, and/or one or morecomponent of the network of servers/backend devices 220). In someembodiments, the button for a single button press method to open agarage (e.g., with respect to FIG. 19) may be the button 306 of an A/Vdevice 210.

Although the A/V recording and communication device 210 (or A/V device210) is referred to herein as an “audio/video” device, the A/V device210 need not have both audio and video functionality. For example, insome embodiments, the A/V device 210 may not include the speakers 330,microphones 328, and/or audio CODEC. In such examples, the A/V device210 may only have video recording and communication functionalities. Inother examples, the A/V device 210 may only have the speaker(s) 330 andnot the microphone(s) 328, or may only have the microphone(s) 328 andnot the speaker(s) 330.

FIG. 4 is another functional block diagram illustrating an embodiment ofthe A/V device 210 according to various aspects of the presentdisclosure. In some embodiments, the A/V device 210 may represent, andfurther include one or more of the components from, the A/V recordingand communication doorbell 210(c), the A/V recording and communicationsecurity camera 210(a), and/or the floodlight controller 210(b).Additionally, in some embodiments, the A/V device 210 may omit one ormore of the components shown in FIG. 4 and/or may include one or moreadditional components not shown in FIG. 4.

As shown in FIG. 4, the A/V device 210 includes memory 402, which mayrepresent the volatile memory 322 and/or the non-volatile memory 324.The memory 402 stores a device application 404. In various embodiments,the device application 404 may configure the processor(s) 310 to captureimage data 406 using the camera 314, audio data 408 using themicrophone(s) 328, input data 410 using the button 306 (and/or thecamera 314 and/or the motion sensor(s) 326, depending on theembodiment), and/or motion data 412 using the camera 314 and/or themotion sensor(s) 326. In some embodiments, the device application 404may also configure the processor(s) 310 to generate text data 414describing the image data 406, the audio data 408, and/or the input data410, such as in the form of metadata, for example.

In various embodiments, the device application 404 may configured theprocessor(s) 310 to generate, store, and/or use data such as garage doorstatus data 420, garage inventory data 422, garage occupancy data 424,garage motion data 426, garage access data 428, garage authorizationdata 430, client device location data 432, or any combination thereof.Some of this data may be determined based image data captured by thecamera 314 and/or the motion sensor(s) 326 (e.g., the image data 406and/or the motion data 412). For example, the garage door status data420 (e.g., whether the garage is open, closed, or somewhere in between),may be determined by capturing image data of a garage door anddetermined its state. The garage inventory data 422 (e.g., an inventoryof objects and/or vehicles in the garage) may be determined by capturingimage data of the contents of a garage and creating an inventory ofthose contents, for example as described with respect to FIGS. 1B, 1Cand 12 herein. The garage occupancy data 424 may indicate whether anypersons are present in and/or around the garage, and may also bedetermined based on captured image data. The garage motion data 426 mayalso be determined based on captured image data. The garage motion data426 includes information stored about any movement captured in or arounda garage, including movement of people, vehicles, objects, the garagedoor itself, and any other movement captured by the camera 314 and/orthe motion sensor(s) 326. The garage access data 428 may also bedetermined based on captured image data. The garage access data 428 is alist that ties together some of the garage motion data 426 withinformation from the garage authorization data 430. The garage accessdata 428 is a log that keeps track of who has accessed a garage andwhen. For example, if a garage door is open for a car in the drivewaythat is recognized as an authorized vehicle, the time, date, and imagesof that car accessing the driveway may be stored as garage access data428. An entry in the garage access data 428 may also include or link toinformation about that vehicle and/or party that accessed the garage.For example, the name of the party linked to an authorized vehicle maybe saved along with the access data for that incident of the garagebeing accessed. In this way, a log of who has accessed a garage and whenmay be kept as the garage access data 428.

The garage authorization data 430 may also be adjusted over time. Forexample, a client device may add guest and/or guest vehicle informationto the garage authorization data 430 so that guests may access a garage,such as described with respect to FIG. 13. A client device may alsogenerate and/or receive scannable codes that allow anyone to access agarage. Information about which scannable codes have been activated andfor how long they are activated may be stored in the garageauthorization data 430, such as described with respect to FIG. 14. Inthis way the AV recording and communication device 210 may determinefrom image data it captures when someone is authorized to access agarage. The client device location data 432 may include informationabout the geolocation of various client devices, including clientvehicle devices. This client device location data 432 may be used asdescribed herein to determine when to allow access to a garage andtransmit an actuation command to an actuator of a garage door. Forexample, if a car that looks like a client's car is in the driveway, butthe client device location data 432 indicates that a client vehicledevice associated with the client's car is 20 miles away, the system maynot grant access to the garage.

In addition, the device application 404 may configure the processor(s)310 to transmit the image data 406, the audio data 408, the motion data412, the input data 410, the text data 414, message(s) 416, the garagedoor status data 420, the garage inventory data 422, the garageoccupancy data 424, the garage motion data 426, the garage access data428, the garage authorization data 430, and/or the client devicelocation data 432 to the client devices 214, 216, 230, the hub device202, and/or the backend server 224 using the communication module 312.In various embodiments, the device application 404 may also configurethe processor(s) 310 to generate and transmit an output signal 418 thatmay include the image data 406, the audio data 408, the text data 414,the input data 410, the motion data 412, the garage door status data420, the garage inventory data 422, the garage occupancy data 424, thegarage motion data 426, the garage access data 428, the garageauthorization data 430, and/or the client device location data 432. Insome of the present embodiments, the output signal 418 may betransmitted to the backend server 224 and/or the hub device 202 usingthe communication module 312. The backend server 224 may then transmit(or forward) the output signal 418 to the client device(s) 214, 216,230, and/or the hub device 202 may then transmit (or forward) the outputsignal 418 to the client device(s) 214, 216, 230, and/or the hub device202 may then transmit (or forward) the output signal 418 to the backendserver 224, and the backend server 224 may then transmit (or forward)the output signal 418 to the client device(s) 214, 216, 230. In otherembodiments, the output signal 418 may be transmitted directly to theclient device(s) 214, 216, 230 by the A/V device 210.

In further reference to FIG. 4, the image data 406 may comprise imagesensor data such as (but not limited to) exposure values and dataregarding pixel values for a particular sized grid. The image data 406may include still images, live video, and/or pre-recorded images and/orvideo. The image data 406 may be recorded by the camera 314 in a fieldof view of the camera 314.

In further reference to FIG. 4, the motion data 412 may comprise motionsensor data generated in response to motion events. For example, themotion data 412 may include an amount or level of a data type generatedby the motion sensor(s) 326 (e.g., the voltage level output by themotion sensor(s) 326 when the motion sensor(s) 326 are PIR type motionsensor(s)). In some of the present embodiments, such as those where theA/V device 210 does not include the motion sensor(s) 326, the motiondata 412 may be generated by the camera 314. In such embodiments, basedon a frame by frame comparison of changes in the pixels from the imagedata 406, it may be determined that motion is present.

The input data 410 may include data generated in response to an input tothe button 306. The button 306 may receive an input (e.g., a press, atouch, a series of touches and/or presses, etc.) and may generate theinput data 410 in response that is indicative of the type of input. Inembodiments where the A/V device 210 is not a doorbell (e.g., the videodoorbell 210(c)), the A/V device 210 may not include the button 306, andthe A/V device 210 may not generate the input data 410.

With further reference to FIG. 4, a message 416 may be generated by theprocessor(s) 310 and transmitted, using the communication module 312, tothe client device 214, 216, 230, the backend server 224, and/or the hubdevice 202. For example, in response to detecting motion using thecamera 314 and/or the motion sensor(s) 326, the A/V device 210 maygenerate and transmit the message 416. In some of the presentembodiments, the message 416 may include at least the image data 406,the audio data 408, the text data 414, the motion data 412, the garagedoor status data 420, the garage inventory data 422, the garageoccupancy data 424, the garage motion data 426, the garage access data428, the garage authorization data 430, and/or the client devicelocation data 432.

As described herein, the message(s) 416 may include messages, signals,data, notifications, and/or any type of electronic communication thatelectronic devices (e.g., the A/V device 210, the client device 214,216, 230, the hub device 202, and/or one or more components of thenetwork(s) of servers/backend devices 220) may transmit and receive withother electronic devices (e.g., the A/V device 210, the client device214, 216, 230, the hub device 202, and/or one or more components of thenetwork(s) of servers/backend devices 220). For instance, message(s) 416may include push notifications, email messages, short message service(SMS) messages, multimedia messages (MMS), voicemail messages, videosignals, audio signals, data transmissions, and/or any other type ofelectronic communication that an electronic device may send to anotherelectronic device.

The image data 406, the audio data 408, the text data 414, the motiondata 412, the garage door status data 420, the garage inventory data422, the garage occupancy data 424, the garage motion data 426, and/orthe garage access data 428 may be tagged with (e.g., a time stamp, basedon clock data) and/or stored separately (e.g., on the backend server224, the hub device 202, and/or the A/V device 210) based on when themotion was detected, how long the motion was detected for, and/or aduration of time associated with the detected motion, or motion event(e.g., the duration of time may include the time the motion was detectedplus an additional time, such as, without limitation, 5 seconds, 10seconds, or 30 seconds). For example, each separate detection of motion,or motion event, may be associated with image data 406, audio data 408,text data 414, motion data 412, garage door status data 420, garageinventory data 422, garage occupancy data 424, garage motion data 426,and/or garage access data 428 representative of the detection of motion,or motion event. As a result, when a request for data pertaining toparticular motion event, or a particular time period, is received (e.g.,by the client device 214, 216, 230, the backend server 224, and/or thehub device 202), the image data 406, the audio data 408, the text data414, the motion data 412, the garage door status data 420, the garageinventory data 422, the garage occupancy data 424, the garage motiondata 426, and/or the garage access data 428 associated with a particularmotion event, and/or associated with motion event(s) within theparticular time period, may be transmitted, retrieved, and/or received.

Although examples discuss the A/V device 210 generating and transmittingthe image data 406, the audio data 408, the text data 414, the motiondata 412, the garage door status data 420, the garage inventory data422, the garage occupancy data 424, the garage motion data 426, and/orthe garage access data 428 when motion is detected (e.g., in the message416), in other examples the data may be generated and/or transmitted atother times. For example, the image data 406, the audio data 408, thetext data 414, the motion data 412, the garage door status data 420, thegarage inventory data 422, the garage occupancy data 424, the garagemotion data 426, and/or the garage access data 428 may be generated andtransmitted continuously (e.g., in a streaming manner), periodically,upon request, etc. In examples where the image data 406, the audio data408, the text data 414, the motion data 412, the garage door status data420, the garage inventory data 422, the garage occupancy data 424, thegarage motion data 426, and/or the garage access data 428 may begenerated and transmitted continuously, the detection of motion (e.g., amotion event) may cause an indication of when the motion was detected(e.g., a time stamp) and/or how long the motion was detected for (e.g.,a duration) to be associated with the image data 406, the audio data408, the text data 414, the motion data 412, the garage door status data420, the garage inventory data 422, the garage occupancy data 424, thegarage motion data 426, and/or the garage access data 428. As a result,even though the image data 406, the audio data 408, the text data 414,the motion data 412, the garage door status data 420, the garageinventory data 422, the garage occupancy data 424, the garage motiondata 426, and/or the garage access data 428 may be continuouslygenerated by the A/V device 210, the image data 406, the audio data 408,the text data 414, the motion data 412, the garage door status data 420,the garage inventory data 422, the garage occupancy data 424, the garagemotion data 426, and/or the garage access data 428 associated withmotion events may be tagged and/or stored separately (e.g., similar tothat of the image data 406, the audio data 408, the text data 414,and/or the motion data 412 generated in response to the detection ofmotion), from the image data 406, the audio data 408, the text data 414,the motion data 412, the garage door status data 420, the garageinventory data 422, the garage occupancy data 424, the garage motiondata 426, and/or the garage access data 428 that is not associated withmotion events.

As described herein, at least some of the processes of the backendserver 224, the hub device 202, and/or the client device 214, 216, 230may be executed by the A/V device 210. In various embodiments, any ofthe processes shown and described with respect to FIGS. 8-23 may beexecuted by any of the backend server 224, the hub device 202, theclient device 214, 216, 230, the A/V device 210, or any combinationthereof.

FIG. 5 is a functional block diagram illustrating one embodiment of thebackend server 224 according to various aspects of the presentdisclosure. The backend server 224 may comprise processor(s) 502 (whichmay be similar to, and/or include similar functionality as, theprocessor(s) 310), a communication module 504 (which may be similar to,and/or include similar functionality as, the communication module 312),and a memory 506 (which may be similar to, and/or include similarfunctionality as, the memory 402). The communication module 504 mayallow the backend server 224 to access and communicate with devicesconnected to the network (Internet/PSTN) 212 (e.g., the A/V device 210,the hub device 202, the client devices 214, 216, 230, a devicecontrolled by the security monitoring service 228, the A/V devices 230,and/or the client devices 232).

The backend server 224 may store, similar to the A/V device 210, garagedoor status data 420, garage inventory data 422, garage occupancy data424, garage motion data 426, garage access data 428, garageauthorization data 430, and/or client device location data 432. Thisinformation may be received from A/V devices 210, and may be usedaccording to the embodiments described herein. For example, the backendserver 224 may determine when a garage door should be opened, andtransmit an actuation command to a garage door actuator either directlyor indirectly. That determination to transmit the actuation command maybe based on the garage authorization data 430, the client devicelocation data 432, and/or any other information as described herein. Thegarage door status data 420, the garage inventory data 422, the garageoccupancy data 424, the garage motion data 426, and/or the garage accessdata 428 may be stored on the backend server 224 so that it is easilyaccessible/requestable by client devices 214, 216, 230, smart-home hubdevices 202, and/or any other device.

The memory 402 may also include a server application 508 that configuresthe processor(s) 502 to receive and/or retrieve the audio data 408, thetext data 414, the input data 410, the messages 416, the image data 406,the motion data 412, the garage door status data 420, the garageinventory data 422, the garage occupancy data 424, the garage motiondata 426, the garage access data 428, the garage authorization data 430,and/or the client device location data 432 from the A/V device 210(e.g., in the output signal 418) and/or the hub device 202. The serverapplication 508 may also configure the processor(s) 502 to transmit(and/or forward) the audio data 408, the text data 414, the input data410, the messages 416, the image data 406, the motion data 412, thegarage door status data 420, the garage inventory data 422, the garageoccupancy data 424, the garage motion data 426, the garage access data428, the garage authorization data 430, and/or the client devicelocation data 432 to the client devices 214, 216, 230 using thecommunication module 504. Furthermore, the server application 508 mayconfigure the processor(s) 502 to receive, using the communicationmodule 504, image data 512 (also referred to as “second image data 512”)generated by the A/V devices 230.

Although referred to as the backend server 224 with reference to theprocesses described herein, the backend server 224 may additionally, oralternatively, include one or more of the devices from the network(s) ofservers/backend devices 220. For example, the processes described hereinwith respect to the backend server 224 may additionally, oralternatively, at least in part, be performed by one or more backendAPIs 226.

In further reference to FIG. 5, the memory 506 may also include sourceidentifying data 510 that may be used to identify the A/V device 210,the hub device 202, and/or the client devices 214, 216, 230. Inaddition, the source identifying data 510 may be used by theprocessor(s) 502 of the backend server 224 to determine the clientdevices 214, 216, 230 are associated with the A/V device 210 and/or thehub device 202.

In some embodiments, the server application 508 may further configurethe processor(s) 502 to generate and transmit a report signal (notshown) to a third-party client device (e.g., electronic device(s) 234),which may be associated with a law enforcement agency or the securitymonitoring service 228, for example. The report signal, which may be themessage 416, in some examples, may include the image data 406, the audiodata 408, the text data 414, the second image data 512, the garage doorstatus data 420, the garage inventory data 422, the garage occupancydata 424, the garage motion data 426, the garage access data 428, thegarage authorization data 430, and/or the client device location data432.

As described herein, at least some of the processes of the A/V device210, the hub device 202, and/or the client device 214, 216, 230 may beexecuted by the backend server 224. In various embodiments, any of theprocesses shown and described with respect to FIGS. 8-23 may be executedby any of the backend server 224, the hub device 202, the client device214, 216, 230, the A/V device 210, or any combination thereof.

For example, the server application 508 may configure the processor(s)502 to analyze the image data 406 in order to determine if the imagedata 406 depicts an object. Objects may include, but are not limited to,people, animals, vehicles, parcels (e.g., packages), electronic devices(e.g., remote control vehicles, drones, etc.), and/or any other type ofobject that may be depicted by the image data 406 and/or cause motionthat may be detected by the A/V device 210. In some examples, theprocessor(s) 502 of the backend server 224 may analyze the image data406 whenever the backend server 224 receives the image data 406 from theA/V device 210.

In some examples, to analyze the image data 406, computer visionprocessing and/or image processing, as described herein, for example,may be performed by the processor(s) 502 of the backend server 224 todetermine that the image data 406 depicts one or more objects. Forexample, in any of the present embodiments, the image data 406 generatedby the A/V device 210 may be analyzed to determine object data 512, thegarage door status data 420, the garage inventory data 422, the garageoccupancy data 424, the garage motion data 426, and/or the garage accessdata 428. In some of the present embodiments, one or more of the imagedata 406, the motion data 412, and the audio data 408 may be used todetermine the object data 512, the garage door status data 420, thegarage inventory data 422, the garage occupancy data 424, the garagemotion data 426, and/or the garage access data 428. The computer visionand/or image processing may be executed using computer vision and/orimage processing algorithms. Examples of computer vision and/or imageprocessing algorithms may include, without limitation, spatial gesturemodels that are 3D model-based and/or appearance based. 3D model-basedalgorithms may include skeletal and volumetric, where volumetric mayinclude NURBS, primitives, and/or super-quadrics, for example.

In some embodiments, the processor(s) 502 of the backend server 224 maycompare the object data 512 to an object database 514 to determine what,if any, object(s) the image data 406 depicts in the field of view of theA/V device 210. For example, the object database 514 may store imagedata corresponding to images and/or video footage that depict variousobjects, where the image data may be labeled (e.g., tagged, such as inthe form of metadata) to indicate an object type 516 (alternativelyreferred to herein as the “type of object 516”) depicted by each imageand/or video footage. For a first example, the object database 514 maystore image data depicting a person, where the image data is labeled toindicate that the type of object 516 includes a person. For a secondexample, the object database 514 may store image data depicting ananimal (e.g., a dog, a cat, a coyote, etc.), where the image data islabeled to indicate that the type of object 516 includes the animal(e.g., the dog, the cat, the coyote, etc.). For a third example, theobject database 514 may store image data depicting a vehicle, where theimage data is labeled to indicate the type of object 516 includes thevehicle.

Based on the comparing, the processor(s) 502 of the backend server 224may match the object data 512 from the image data 406 to the image datastored in the object database 514. The processor(s) 502 of the backendserver 224 may then use the match to determine that the object data 512represents an object and/or to determine the type of object 516 that theobject data 512 represents. For example, if the processor(s) 502 of thebackend server 224 matches the object data 512 from the image data 406to image data stored in the object database 514 that represents aperson, then the processor(s) 502 of the backend server 224 maydetermine that the image data 406 depicts an object and/or that theimage data 406 depicts a person. In some examples, when the object data512 represents multiple objects, the processor(s) 502 of the backendserver 224 may perform a similar analysis to identify each objectrepresented by the object data 512 and/or the respective type of object516 associated with each of the objects represented by the object data512.

In some examples, in addition to, or alternatively from, comparing theimage data 406 to the image data stored in the object database 514,features and/or characteristics of various objects may be stored in theobject database 514, and the features and/or characteristics of theobjects in the image data 406 may be determined (e.g., using computervision processing, image processing, or the like) and compared againstthe features and/or characteristics from the object database 514. Forexample, sizes, volumes, weights, colors, movement types, and/or otherfeatures and/or characteristics of various objects may be stored in theobject database 514. The size, volume, weight, color, movement type,and/or other features and/or characteristics of an object depicted bythe image data 406 may then be compared to the sizes, volumes, weights,colors, movement types, and/or other features and/or characteristicsstored in the object database 514 to identify the type of object 516depicted by the image data 406.

Comparing the image data 406 to the image data stored in the objectdatabase 514 may inform information stored as the garage door statusdata 420, the garage inventory data 422, the garage occupancy data 424,the garage motion data 426, and/or the garage access data 428. Forexample, if a vehicle is recognized and accesses the garage, thatinformation may be stored as garage motion data 426 and/or garage accessdata 428. If an object within a garage is moved, that object may berecognized and the removal of that object may be stored as garageinventory data 422. If a garage door opening is recognized, that may bestored as garage door status data 420.

Although described as being performed in the backend server 224, in someembodiments, the image data 406 may be analyzed by any of the ANrecording and communication device 210, the hub device 202, and/or theclient device 214, 216, 230 in order to determine if the image data 406depicts an object, therein. Thus, any or all of the operations describedherein to analyze the image data 406 may be performed by any of thesedevices. To perform these operations, any or all of these devices mayalso include the object database 514, including the object type 516,and/or the object data 514, as described with reference to FIG. 5.

The hub device 202 and/or the backend server 224 (and/or one or moreadditional or alternative components of the network(s) ofservers/backend devices 220) may alternatively be referred to herein as“network devices.”

Now referring to FIGS. 6A and 6B, FIGS. 6A and 6B are a functional blockdiagrams illustrating one embodiment of the client devices 214, 216, 230according to various aspects of the present disclosure. The clientdevices 214, 216, 230 may comprise processor(s) 602 (which may besimilar to, and/or include similar functionality as, the processor(s)310) that are operatively connected to an input interface 604,microphone(s) 606, speaker(s) 608, a communication module 610 (which maybe similar to, and/or include similar functionality as, thecommunication module 312), and memory 612 (which may be similar to,and/or include similar functionality as, the memory 402). The clientdevice 214, 216, 230 may further comprise a camera (not shown)operatively connected to the processor(s) 602. The client devices 214,216 may be, for example, a smart phone, tablet, or laptop computer. Theclient vehicle device 230 may be, for example, a computing systemon-board a vehicle.

The memory 612 may store a device application 614. In variousembodiments, the device application 614 may configure the processor(s)602 to receive input(s) to the input interface 604 (e.g., a request toopen or close a garage door, a request to generate a scannable code, arequest to add a guest license plate to the garage authorization data430). In addition, the device application 614 may configure theprocessor(s) 602 to receive, using the communication module 610, theinput data 410, the image data 406, the audio data 408, the outputsignal 418, messages 416, the garage door status data 420, the garageinventory data 422, the garage occupancy data 424, the garage motiondata 426, the garage access data 428, the garage authorization data 430,and/or the client device location data 432 from one or more of the A/Vdevice 210, the hub device 202, or the backend server 224.

With further reference to FIG. 6, the input interface 604 may include adisplay 618. The display 618 may include a touchscreen, such that theuser of the client device 214, 216, 230 may provide inputs directly tothe display 618 (e.g., a request to open or close a garage door, arequest to generate a scannable code, a request to add a guest licenseplate to the garage authorization data 430). In some embodiments, theclient device 214, 216, 230 may not include a touchscreen. In suchembodiments, and in embodiments where the client device 214, 216, 230includes the touchscreen, the user may provide an input using any inputdevice, such as, without limitation, a mouse, a trackball, a touchpad, ajoystick, a pointing stick, a stylus, etc.

In some of the present embodiments, in response to receiving a message416, the device application 614 may configure the processor(s) 602 tocause the display 618 to display the message 416. The message 416 mayindicate that the A/V device 210 detected motion (e.g., of a garagedoor, in a garage, in a driveway, in an area about a garage), detectedthe presence of an object (e.g., a vehicle, an object as shown in FIG.1B or 1C), received an input (e.g., to the button 306, to the keypad156), etc. While displaying the message 416, the input interface 604 mayreceive input from the user to answer the message 416. In response, thedevice application 614 may configure the processor(s) 602 to display thereceived image data 406 on the display 618 (e.g., display image(s)and/or video footage represented by the image data 406).

The client vehicle device 230 also includes a garage door controlinterface 632 and an on-board diagnostic (OBD) interface 634. A garagedoor control interface may be displayed on the display 618 of any of theclient devices 214, 216, 230, and interacted with via the inputinterface 604 (e.g., to transmit an actuation command to an actuator ofa garage door). However, the client vehicle device 230 may also have thegarage door control interface 632, which is a separate interface forcontrolling a garage, gate, lights, or other programmable aspects of asmart home. For example, the garage door control interface 632 mayinclude physical buttons on the interior of a car, such as in the dash,in a sun visor, and/or in a rear-view mirror. These physical buttons maybe used to actuate a garage door, and the button presses may further beused to determine garage motion data 426, the cause of observed garagemotion (e.g., who or what vehicle opened the garage door, which may bestored as garage access data 428), and/or garage door status data 420(e.g., whether door is closed—if button on the garage door controlinterface is pressed the garage door status 420 may be changedaccordingly).

The OBD interface 634 is connected to status sensors and otherelectronic devices of a vehicle. For example, inputs from a vehicle tothe OBD interface 634 may include any metric or aspect of a car measuredby OBD systems. Such information may be used to determine, for example,if a car is running or not. Information from the OBD interface 634 maybe used in various embodiments as described herein. For example, beforedetermining that a garage door should be closed after a vehicle pullsinto the garage, the system may wait until the vehicle's engine has beenshut off. In this way, emissions from the vehicle will not be trappedinside the garage. Information from the OBD interface 634 may indicatewhether the vehicle's engine is still running or not. In variousembodiments, information from the OBD interface 634 may also be usedaccording to the process shown and described with respect to FIG. 16.

As described herein, at least some of the processes of the A/V device210, the hub device 202, and/or the backend server 224 may be executedby the client device 214, 216, 230. In various embodiments, any of theprocesses shown and described with respect to FIGS. 8-23 may be executedby any of the backend server 224, the hub device 202, the client device214, 216, 230, the A/V device 210, or any combination thereof.

FIG. 7 is a functional block diagram illustrating an embodiment of thesmart-home hub device 202 (alternatively referred to herein as the “hubdevice 202”) according to various aspects of the present disclosure. Thehub device 202 may be, for example, one or more of a Wi-Fi hub, asmart-home hub, a hub of a home security/alarm system, a gateway device,a hub for a legacy security/alarm system (e.g., a hub for connecting apre-existing security/alarm system to the network (Internet/PSTN) 212for enabling remote control of the hub device 202), and/or anothersimilar device. In some examples, the hub device 202 may include thefunctionality of the VA device 208. The hub device 202 may compriseprocessor(s) 702 (which may be similar to, and/or include similarfunctionality as, the processor(s) 310) that are operatively connectedto speaker(s) 704, microphone(s) 706, a communication module 708 (whichmay be similar to, and/or include similar functionality as, thecommunication module 310), and memory 710 (which may be similar to,and/or include similar functionality as, the memory 402). In someembodiments, the hub device 202 may further comprise one or more of acamera (not shown). In some embodiments, the hub device 202 may notinclude one or more of the components shown in FIG. 7, such as thespeaker(s) 704 and/or the microphone(s) 706.

As shown in the example of FIG. 7, the memory 710 stores a smart-homehub application 712. In various embodiments, the smart-home hubapplication 712 may configure the processor(s) 702 to receive sensordata from the sensors 204 and/or the automation devices 206. Forexample, the sensor data may include a current state (e.g.,opened/closed for door and window sensors, motion detected for motionsensors, living room lights on/off for a lighting automation system,garage door open/closed etc.) of each of the sensors 204 and/or theautomation devices 206. In some of the present embodiments, the sensordata may be received in response to sensor triggers. The sensor triggersmay be a door opening/closing, a window opening/closing, lights beingturned on/off, blinds being opened/closed, garage door opening/closingetc. As such, the sensor data may include the current state of thesensors 204 and/or the automation devices 206 as well as any updates tothe current state based on sensor triggers.

With further reference to FIG. 7, the smart-home hub application 712 mayconfigure the processor(s) 702 to receive the audio data 408, the textdata 414, the image data 406, the motion data 412, the input data 410,the messages 416, the garage door status data 420, the garage inventorydata 422, the garage occupancy data 424, the garage motion data 426, thegarage access data 428, the garage authorization data 430, and/or theclient device location data 432 from the A/V device 210 and/or clientdevices 214, 216, 230 (in some embodiments, via the backend server 224)using the communication module 708. For example, the hub device 202 mayreceive and/or retrieve (e.g., after receiving a signal from the A/Vdevice 210 that the A/V device 210 has been activated) the image data406, the input data 410, the motion data 412, the garage door statusdata 420, the garage inventory data 422, the garage occupancy data 424,the garage motion data 426, the garage access data 428, the garageauthorization data 430, and/or the client device location data 432 fromthe A/V device 210 and/or the backend server 224 in response to motionbeing detected by the A/V device 210. The smart-hub application 712 maythen configure the processor(s) 702 to transmit, using the communicationmodule 708, the audio data 408, the text data 414, the image data 406,the motion data 412, the input data 410, the messages 416, the garagedoor status data 420, the garage inventory data 422, the garageoccupancy data 424, the garage motion data 426, the garage access data428, the garage authorization data 430, and/or the client devicelocation data 432 to the client device 214, 216, 230, the backend server224, and/or an additional electronic device (e.g., a second A/V device210, the automation device(s) 206, the sensor(s) 204, etc.).

As described herein, at least some of the processes of the A/V device210, the backend server 224, and/or the client device 214, 216, 230 maybe executed by the hub device 202. In various embodiments, any of theprocesses shown and described with respect to FIGS. 8-23 may be executedby any of the backend server 224, the hub device 202, the client device214, 216, 230, the A/V device 210, or any combination thereof.

Each of the processes described herein, including the processes 800,900, 1000, 1100, 1200, 1300, 1400, 1500, 1600, 1700, 1800, 1900, 2000,2100, 2200, and 2300, are illustrated as a collection of blocks in alogical flow graph, which represent a sequence of operations that may beimplemented in hardware, software, or a combination thereof. In thecontext of software, the blocks represent computer-executableinstructions stored on one or more computer-readable storage media that,when executed by one or more processors, perform the recited operations.Generally, computer-executable instructions include routines, programs,objects, components, data structures, and the like that performparticular functions or implement particular abstract data types. Theorder in which the operations are described is not intended to beconstrued as a limitation, and any number of the described blocks may becombined in any order and/or in parallel to implement the processes.Additionally, any number of the described blocks may be optional andeliminated to implement the processes.

FIG. 8 is a flowchart illustrating an example process 800 for detectinga vehicle and opening a garage door, according to various aspects of thepresent disclosure. According to the process 800, image data andwireless communication with an electronic device is used to authorize avehicle's access to a garage. For example, a license plate may becaptured by a camera of an A/V device, and the A/V device maycommunicate with a client vehicle device in the vehicle to checkelectronic credentials. If both a license plate number and theelectronic credentials match authorization/access data stored in amemory, an actuation command is transmitted to cause a garage door toopen. In this way, multiple factors may be used to automaticallyauthorize access to a garage and cause the garage door to open.Advantageously, safety and convenience factors are improved using such aprocess.

At block B802, a vehicle is detected within an area about the garagedoor. For example, the backend server 224 may receive, using thecommunication module 504, image data 406 generated by the A/V device210. The image data 406 may indicate movement of a vehicle within afield of view of the A/V device 210 (e.g., a field of view of the camera314 and/or a field of view of the motion sensor(s) 326). In someexamples, the backend server 224 may receive the image data 406 from theA/V device 210, the hub device 202, and/or the client device 214, 216,230. In some examples, the backend server 224 may receive the image data406 based on the A/V device 210 detecting motion of the vehicle. Thevehicle may be moving toward the garage door (e.g., on a driveway towardthe garage door, pulling into the driveway from a road, etc.). Themovement of the vehicle toward the garage door may be deduced based onthe image data. For example, referring FIG. 1A, once a vehicle is in thedriveway 132 (e.g., the vehicle 130) it may move directly toward garage,which may be deduced from image data captured by the A/V devices 126and/or 104. The system may also be able to intuitively deduce that avehicle is or will soon be moving toward a garage door. For example, thevehicle 136 may drive down the road 134 toward the driveway and slowdown at a rate that suggests it will turn into the driveway 132. Thismotion may be captured through image data and a deduction made that thevehicle 136 will approach the garage. In other words, the system mayconsider the vehicle 136 as within an area about the garage door. Inanother example, the vehicle 136 may stop near the driveway 132, waitingfor oncoming traffic to pass before turning left into the driveway 132.The vehicle also may be deduced to be within an area about the garagedoor in this scenario, based on image data captured by the A/V devices104 and/or 126.

At block B804, image data of an object associated with the vehicle isreceived by a camera of the A/V device. For example, the backend server224 may receive, using the communication module 504, image data 406generated by the A/V device 210. The image data 406 may show an objectassociated with a vehicle that is within a field of view of the A/Vdevice 210 (e.g., a field of view of the camera 314 and/or a field ofview of the motion sensor(s) 326). In some examples, the backend server224 may receive the image data 406 from the A/V device 210, the hubdevice 202, and/or the client device 214, 216, 230. In some examples,the backend server 224 may receive the image data 406 based on the A/Vdevice 210 detecting motion of the vehicle (e.g., detecting the vehicleat the block B802). The object associated with the vehicle may be, forexample, a scannable code on, in, or near the vehicle (e.g., a stickerin the windshield of a vehicle visible from outside the vehicle, a codepresented on the display of a client device), a license plate of thevehicle, a feature of the vehicle, etc.

At block B806, the object is identified. The object may be identified invarious ways, and the identification may be performed by various devicesdescribed herein such as the backend server 224, the A/V device 210, thehub device 202, and/or the client device 214, 216, 230. The object maybe identified by an image recognition process. For example,identification of the object may be based on comparing the receivedimage data of the object with previously stored image data. The clientmay take a picture of a license plate of their vehicle with their clientdevice. This previously stored image of the client's license plate maybe compared to the image data, and a degree of similarity between theimages may be used to determine whether the image data shows the samelicense plate as the previously stored image.

In another example, a previously taken photo of the client's licenseplate may be OCR'd to determine the characters (e.g., license platenumber, issuing state, etc.) of a license plate. Subsequent image datamay also be OCR'd to determine if the characters on the previously OCR'dlicense plate information matches the license plate of the image data.In another example, the client may enter input into a client device thestate name and exact characters of their license plate. In this way, theimage data may be OCR'd to determine if a vehicle detected about agarage has a license plate that matches the license plate informationentered into a client device. In another example, the object identifiedin the image data is a machine-readable optical label. For example, a QRcode, bar code, or other scannable visual code may be affixed to thevehicle, in the vehicle, or otherwise associated with the vehicle. Themachine-readable optical label may be identified as the object from theimage data, and the code may then be read from the image data. In someembodiments, a scannable code associated with a vehicle may be presentedby a person on a client device. For example, a QR code for accessing agarage may be presented on the display of a client device so that it maybe captured by a camera of an A/V device.

At block B808, an electronic device associated with the vehicle iswirelessly communicate with. The electronic device may be in, on, ornear the vehicle. For example, the electronic device may be a clientvehicle device 230 in the vehicle. The electronic device may be a clientdevice 214/216 in or near the vehicle. In some embodiments, near thevehicle may mean that the electronic device is within a predetermineddistance of the vehicle (e.g., within 1 meter, 3 meters, 10 meters, 20meters, 50 meters, etc.). In some embodiments near the vehicle may meanthat the electronic device is in the field of view of the A/V devicethat has captured image data of the vehicle (e.g., for electronicdevices for which communication uses a line-of-sight). In someembodiments, near the vehicle may mean that the electronic device isnear enough to the client vehicle device and/or the A/V device thatelectronic device may successfully communicate. For example, if NFC orBLE devices are used, they may communicate over a limitedrange/distance. Thus, the electronic device must be near enough towhatever it is communicating with to actually communicate. For example,an A/V (or a smart-home hub device, a client device, a backend server,etc.) device may scan an NFC device to wirelessly communicate accordingto the block B808. As another example, an electronic device maycommunicate with a vehicle client device, which may facilitatecommunication between the electronic device and an A/V device (or asmart-home hub device, a client device, a backend server, etc.).

In some embodiments, the wireless communication with the electronicdevice at block B808 may occur in response to or based on theidentification of the object. In other words, the wireless communicationmay not be initiated until the object is identified at block B806. Forexample, an A/V device may not attempt to communicate with an electronicdevice regarding garage access until object data that is recognized bythe system is identified. The communication may not be attempted until,for example, a license plate or scannable visual code affixed to avehicle is recognized in image data. This may add an extra layer ofsecurity, because wireless signals seeking to communicate with anelectronic device are not broadcast (and therefore could not be listenedto or picked up easily by unauthorized parties) without the initialvisual identification of the object. Once the object is identified, thewireless communication is initiated.

In various embodiments, the wireless communication is for authenticatingthe electronic device. In other words, if proper credentials arecommunicated by the electronic device, the electronic device isauthenticated. This authenticated communication may be used to determinethat the vehicle is an authorized vehicle as described below withrespect to block B810.

At block B810, the vehicle is determined to be an authorized vehicle.For example, the backend server 224 may receive, using the communicationmodule 504, authentication information from the wireless communicationof block B808 from the A/V device 210. The authentication information inthe wireless communication may come from, for example, a client device214, 216, 230 and be routed through the A/V device 210. Thedetermination that the vehicle is authorized may be based on multiplefactors. For example, the image data of the vehicle may be used toconfirm certain visual aspects of or associated with a vehicle (e.g.,correct license plate information, correct color, make, model ofvehicle, correct scannable code present on window, client device, etc.).The wireless communication from the electronic device may also beauthenticated to determine that the vehicle is an authorized vehicle. Insome embodiments, the authorization (including the wirelesscommunication) may occur automatically. In this way, the vehicle may beauthorized without action from a user such as entering a passcode orpressing a button.

In various embodiments, authentication of the vehicle based on wirelesscommunications with the electronic device may occur in various ways. Forexample, a set of attributes may be received from the electronic devicewithin the vehicle, and those attributes may be compared to previouslystored attributes associated with a set of electronic devices. The setof electronic devices may have been previously set up by a client orotherwise added to a list of electronic devices that are authorized toaccess a garage. The wireless communication to authorize the electronicdevice may be via a short-range wireless communication protocol such asNFC, BLE, or any other suitable short-range communication protocol.

At block B812, an actuation command is transmitted to an actuator of thegarage door to cause the garage door to open. The transmission of theactuation command is in response to/based on a determination that thevehicle is authorized B810. In this way, a vehicle may pull up to agarage and have it open if the vehicle is authorized. Advantageously,the garage may open without any apparent volitional act from a person inthe vehicle. For example, the person in the vehicle may not press abutton, enter a passcode, or otherwise act to make the garage open.Instead, the vehicle approaches the garage and the door just opens, allwhile advantageously identifying an object and wirelessly authenticatingthe vehicle to improve safety. In an example, the backend server 224 maymake the determination that the vehicle is authorized, and may send theactuation command to the actuator that is routed through a network, aclient device 214, 216, 230, smart-home hub device 202, and/or the A/Vdevice 210.

In some embodiments, the vehicle accessing the garage may belong to oneof a variety of parties. For example, the vehicle may belong to an owner(or member of the owner's family) of a home/land that includes thegarage. The vehicle may belong to someone otherwise occupying the land(e.g., lessee, renter, guest, etc.). In some embodiments, the vehiclemay be a service person or delivery driver. In such instances, thevehicle may not be pulled into the garage, but the garage door may opento provide the service person or delivery driver with access to thegarage. If a service person or delivery driver does try to pull theirvehicle into the garage, an alert may be generated and sent to theclient device 214, 216, 230 (or any other device). Access to a garagemay be provided to a delivery driver so that a delivery may be placed inthe garage. Once the delivery driver pulls away the garage may closeagain, based on the image data captured by the A/V device 210. Access toa garage may be provided to a service person so that the service personmay access the interior of a home that is connected to the garage.

FIG. 9 is a flowchart illustrating an example process 900 fortransmitting audible information to a speaker, according to variousaspects of the present disclosure. At block B902, a wirelesscommunication channel is established with a speaker. The connection isestablished so that audio may be streamed to the speaker. In analternative embodiment, a prerecorded audio clip may play from a devicewith a speaker, so that a wireless communication channel need not beestablished. In another alternative embodiment, an audio file may betransmitted to a device with a speaker. The audio file may be played onthe speaker after it has fully transmitted to the device with thespeaker. In this way, a wireless communication channel for streaming maynot be established, but a wireless communication channel fortransmitting the audio file may be established.

In various embodiments, the speaker may be the speaker 330 of the A/Vdevice 210, a speaker of a client device 214, 216, 230, a speaker of asmart-home hub device 202, a standalone speaker, a speaker of a soundsystem of a vehicle, or any other device with a speaker. In embodimentswhere a communication channel is established, the communication may bebetween the speaker or device with a speaker and the A/V device 210, theclient device 214, 216, 230, the smart-home hub device 202, the backendserver 224, or any other electronic device.

In various embodiments, the speaker may be associated with a vehicle,such as a speaker of the vehicle's internal sound system. In this way, aperson inside the vehicle would be able to hear audio played on thespeaker. In various embodiments, a wireless connection may beestablished before or after the garage door is opened, or while thegarage door is opening. Similarly, any audio message played on thespeaker may also begin playing before or after the garage door isopened, or while the garage door is opening.

In various embodiments, instructions relating to an object may also betransmitted to a speaker. For example, the speaker 330 of the A/V device210 in a garage may instruct a person where to place a particular objectwithin the garage. For example, if an object (e.g., lawnmower) has beenremoved from the garage and is now being put back, the instructions mayinstruct on how and where to put the lawnmower back in the place it wasbefore. In another example, a delivery driver may be instructed on whereto leave a package in the garage. This may, for example, prevent apackage from being left in a location where a car typically parks.

At block B904, a signal is transmitted to the speaker to cause thespeaker to emit audio instructions to a driver of the vehicle to movethe vehicle to a predetermined position. The signal with the audioinstructions/message may be transmitted to the speaker from the A/Vdevice 210, the client device 214, 216, 230, the smart-home hub device202, and/or the backend server 224. The speaker may be the speaker 330of the A/V device 210, a speaker of a client device 214, 216, 230, aspeaker of a smart-home hub device 202, a standalone speaker, a speakerof a sound system of a vehicle, or any other device with a speaker. Forexample, when a vehicle is pulling into a garage, a signal may betransmitted to a speaker inside the vehicle that is connected to or partof the client vehicle device 230. The audio instructions played over thespeaker based on the signal may instruct the driver where to park, toslow down, to speed up, to move farther, to back up, to go out of thegarage and pull into another spot, to get out of the vehicle to movesomething out of a parking space in the garage, indicate how close thevehicle is to a wall, indicate how much further the vehicle should move,whether the vehicle (e.g., the steering wheel) should turn to the rightor left, etc. These audio instructions may be determined based on imagedata captured by the A/V device 210 (e.g., in a field of view of thecamera 314 and/or in a field of view of the motion sensor(s) 326). Theseaudio instructions may also be determined based on a computer visionprocess.

The audio instructions may further be determined based on variousfactors, including for example user/client inputs and historical imagedata. For example, the audio instructions instructing a vehicle where topark may be based on a user input/configuration that previouslyindicated where a particular vehicle should park within a garage. Sincevarious embodiments as described herein describe processes forrecognizing particular vehicles, the audio instructions may instruct aparticular vehicle or vehicles where to park within a garage. In anotherexample, the audio instructions may be determined based on historicalimage data captured by the A/V device 210. For example, if the vehicleentering the garage typically parks in a particular place within thegarage, the audio instructions may be configured to assist the driver inreturning the vehicle to that customary location within the garage.

At block B906, a signal is transmitted to the speaker to cause thespeaker to emit audible information about previous actuation commands.The signal transmitting the audible information may be transmitted tothe speaker from the A/V device 210, the client device 214, 216, 230,the smart-home hub device 202, and/or the backend server 224. Thespeaker may be the speaker 330 of the A/V device 210, a speaker of aclient device 214, 216, 230, a speaker of a smart-home hub device 202, astandalone speaker, a speaker of a sound system of a vehicle, or anyother device with a speaker. Previous actuation commands are instancesof a garage door being actuated. For example, information about when agarage door was opened, who it was opened by, and/or how it wasauthorized to be opened may be stored as the garage access data 428. Theinformation transmitted as audible information to a speaker may alsoinclude a time of each actuation, identification of a vehicle associatedwith each actuation, what a person or vehicle did in the garage in thegarage after each actuation, what was moved/added/removed in associationwith each actuation, etc.

Audible information relating to the garage access data 428 is thereforetransmitted to the speaker. For example, as a person pulls into theirgarage when getting home from work, a speaker in their car may informthem: “Your garage was opened twice today, once by your child to gettheir bike out, and once by a delivery driver who left the new TV youordered next to the door into your house.” In this way, informationabout actuation commands may be relayed through a speaker. In such anexample, the delivery driver may be automatically provided with accessto the garage using the processes described herein. The location where apackage was left within the garage may have also been observed by thesystem according to the processes described herein. In another example,the actuation commands information (e.g., information from the garageaccess data 428) may be played on a speaker of a smart-home hub device202 (e.g., after a person walks into their house from the garage). In adelivery driver context, the delivery driver may also be permittedaccess to the garage by scanning a label, code, or electronic deviceaffixed to or otherwise associated with a package. If the code is knownto the system, access to the garage is granted. The system may then alsosend an alert to the client device 214, 216, 230 that their package hasbeen delivered. In one example, a tracking number may be determined froman email receipt from when an item was purchased online. In otherexamples the tracking number of a package may be received from thewebsite on which the package was ordered. Once the tracking number isknown to the system, a code indicating that tracking number may bescanned by presenting the code to the A/V device 210 when the package isbeing delivered. If the code matches the code determined from an emailreceipt or received from the website, the garage door may be opened toallow a delivery driver to place the package inside. The A/V device 210may also capture image data to determine when the delivery driver isdone delivering the package(s), and subsequently send an actuationcommand to close the garage door. In the instance of package delivery, acode associated with a package may only permit access to the garageonce. In this way, a code could not be duplicated to allow access afterthe package has already been delivered. That is, a code associated witha package expires after one use.

At block B908, a signal is transmitted to the speaker to cause thespeaker to emit audible information about security alerts generated by asecurity system. The signal with the audio instructions/message may betransmitted to the speaker from the A/V device 210, the client device214, 216, 230, the smart-home hub device 202, and/or the backend server224. The speaker may be the speaker 330 of the A/V device 210, a speakerof a client device 214, 216, 230, a speaker of a smart-home hub device202, a standalone speaker, a speaker of a sound system of a vehicle, orany other device with a speaker.

In various embodiments, the speaker, A/V device 210, and other devicesof the system (e.g., devices shown in FIG. 2) are connected to asecurity system. The security system may be, for example, the securitymonitoring service 228. An audio announcement played on a speaker may berelated to information received from the security system. For example,if there is a reported break-in, fire, shooting, or any other event inthe area, an audio announcement may be transmitted to a speaker asdescribed herein. In various embodiments, the security system may be asecurity system for the premises on which the garage is located.Accordingly, an audio announcement may include any information relatedto the state of the security system (e.g., whether any alarms have beentriggered, any out of normal range sensor measurements, etc.). In someembodiments, the audio announcement may also be information receivedfrom the security system comprises a number of security alerts generatedby the security system only while the vehicle was outside the garage.

In various embodiments, a signal may be transmitted to a display orother visual indicator such as a light or lights to communicate to adriver of a vehicle or a person generally. The signal may be transmittedin ways similar to the signal of FIG. 9. For example, red and greenlights may be mounted inside the garage that tells the car to keepmoving forward (green) when parking or to stop (red). In anotherexample, information may be communicated to a client vehicle device thathas a display. Security alerts and/or other information may therefore bedisplayed on a screen or interface of the display of a client vehicledevice. The information displayed may include any information that maybe communicated via audio instructions, such as security alerts,actuation commands, and/or instructions to move a vehicle as describedherein.

FIGS. 10 and 11 are flowcharts illustrating example processes 1000 and1100 for detecting a vehicle and closing a garage door, according tovarious aspects of the present disclosure. At block B1002, a garage dooris determined to be open. For example, the A/V device 210 may determinethat a garage door is open based on image data 406 generated by the A/Vdevice 210. The image data 406 may depict a garage door (e.g., thegarage door 162) or an opening (e.g., the opening 116) indicating thatthe garage door is open within a field of view of the A/V device 210(e.g., a field of view of the camera 314 and/or a field of view of themotion sensor(s) 326). The backend server 224 may receive, using thecommunication module 504, the image data 406. In some examples, thebackend server 224 may receive the image data 406 from the A/V device210, the smart-home hub device 202, and/or the client device 214, 216,230. In some examples, the backend server 224 may receive the image data406 based on the A/V device 210 detecting motion.

At block B1004, a vehicle is detected within an area outside a garage.The garage is associated with the garage door. For example, the vehicledetected may be the vehicle 130 as shown in FIG. 1. At block B1006,image data of the vehicle is received by a camera of the A/V device 210.The image data may be the image data 406. At block B1008, the vehicle isdetermined to be moving away from the garage door using the receivedimage data (e.g., the image data 406). Optionally, the vehicle movingaway from the garage may be identified by comparing the received imagedata of the vehicle with previously stored image data of the vehicle. Invarious embodiments, the vehicle may be identified using any of thevarious processes for identifying a vehicle according to the variousembodiments described herein. In this way, the system may be able torecord (e.g., in the garage access data 428), the specific vehicle thathas left (or is leaving) the garage. At block B1010, an actuationcommand is transmitted to an actuator of the garage door to cause thegarage door to close. In various embodiments, the actuation command maybe transmitted after or in response to determining that the vehicle ismoving away from the garage door (block B1008). In various embodiments,the actuation command may be transmitted after or in response to thevehicle being identified.

In various embodiments, the vehicle may be identified by comparing thereceived image data of the vehicle with previously stored image data ofthe vehicle. This process may use a computer vision process. Adetermination that the vehicle is moving away from the garage door mayalso include using a computer vision process on the received image dataof the vehicle. Determining that the vehicle is moving away from thegarage door may also be based on GPS location information or othergeolocation information associated with the vehicle. The GPS and/orgeolocation information may be received at the A/V device 210, thebackend server 224, the smart-home hub device 202, and/or the clientdevice 214, 216 from vehicle via an electronic device in the vehicle(e.g., the client vehicle device 230).

In various embodiments, a garage door that is unmonitored by a human mayinclude an alarm siren and flashing lights. In this manner, the sirenand flashing lights can be used to warn when the garage door is closing.The siren and flashing lights may be triggered based on a signal from anA/V device, which may cause the garage door to move and/or detect thatthe garage door is moving. Other alert methods may also be used toindicate that a garage door is closing, regardless of whether the garagedoor is monitored or not. For example, an alert signal may be sent toclient devices, A/V devices, or any other type of device. The alertsignal may cause a device to transmit an audio alert, a visual alert, asomatosensory alert (e.g., cause a device to vibrate), etc. The alertsignals may be sent to a device or devices that are within apredetermined distance from the garage. Alert signals may be sentaccording to other criteria as well, for example devices that areconnected to a Wi-Fi network of a home attached to the garage, devicesthat are within a field of view of any A/V devices, etc. In variousembodiments, certain devices may be sent an alert when a garage door isopened or closed no matter how close the device is to the garage.

At block B1102, a garage door is determined to be open. Block B1102 maybe similar to the block B1002 of FIG. 10. For example, the A/V device210 may determine that a garage door is open based on image data 406generated by the A/V device 210. The image data 406 may depict a garagedoor (e.g., the garage door 162) or an opening (e.g., the opening 116)indicating that the garage door is open within a field of view of theA/V device 210 (e.g., a field of view of the camera 314 and/or a fieldof view of the motion sensor(s) 326). The backend server 224 mayreceive, using the communication module 504, the image data 406. In someexamples, the backend server 224 may receive the image data 406 from theA/V device 210, the smart-home hub device 202, and/or the client device214, 216, 230. In some examples, the backend server 224 may receive theimage data 406 based on the A/V device 210 detecting motion.

At block B1104, a vehicle is determined to have moved from a firstlocation inside a garage to a second location outside the garage. Thegarage door is associated with the garage (e.g., the garage door 162).The determination may be made using the image data 406 captured by theA/V device 210. At block B1106, an actuation command is transmitted toan actuator of the garage door to cause the garage door to close whenthe vehicle is determined to be outside the garage. The first locationinside the garage and the second location outside the garage may bedetermined relative to the garage door (e.g., the garage door 162)and/or a garage opening (e.g., the opening 116).

In various embodiments, the determination that the garage door is open(e.g., blocks B1002, B1102) is based on receiving one or more readingsfrom one or more sensors associated with the garage door. For example,these one or more sensors may indicate a state of the garage door (e.g.,open, closed, somewhere in between). In these embodiments, the garagedoor may be determined to be open without using the image data 406(e.g., images captured showing the garage door and/or an opening in agarage). Sensors associated with the garage door may also be used todetermine if/when the vehicle is outside the garage (and may be usedinstead of or in combination with determining that the vehicle isoutside the garage based on the image data 406). For example, the sensor160 of FIGS. 1D and 1E may be used to ensure that the vehicle is outsideof the garage before sending an actuation command to close the garagedoor.

In various embodiments, the determination that the vehicle has movedfrom the first location in the garage to the second location outside thegarage may include a determination that the vehicle has moved byperforming a computer vision process on video and/or still images of thevehicle captured by the camera and/or motion sensors of the A/V device210. In various embodiments, the determination that the vehicle hasmoved outside the garage may be based on captured image data of thevehicle over a specified or unspecified time interval. The firstlocation inside the garage and the second location outside the garagemay also be determined, at least in part, based on a specified orunspecified time interval. For example, a vehicle moving out of a garagemay be expected to take at least a minimum amount of time to exit agarage (e.g., based on the size of the garage, a maximum speed at whichthe vehicle may move out of a garage, distance the vehicle must travelto exit the garage, etc.). The system may consider this minimum amountof time when determining whether the vehicle has actually moved outsidethe garage. If it appears that the vehicle is outside of the garage, butthe minimum amount of time has not elapsed since the vehicle was insidethe garage, the system may wait to transmit an actuation signal to thegarage door until it has a higher level of certainty that the vehicle isactually fully outside the garage.

In various embodiments, the determination that the vehicle has movedfrom the first location of the vehicle and the second location of thevehicle may be determined based on comparing a current captured imagedata of the vehicle with a previously captured and stored image data ofthe vehicle. In various embodiments, the determination that the vehiclehas moved from the first location of the vehicle and the second locationof the vehicle may be determined based on GPS location informationreceived from an electronic device located within the vehicle.

In various embodiments, an actuation command may be transmitted to anactuator of a garage door to close the garage door using on similarprocesses and components as described herein with respect to FIG. 8. Forexample, a process for closing a door may include making a determinationthat a garage door is open (e.g., similar to B1002, B1102). The processfurther includes detecting a vehicle within an area outside the garage(e.g., similar to B804, B1004). The process further includes determiningthat the vehicle is moving away from the garage door, for example basedon image data captured by an A/V device (e.g., similar to B1008). Theprocess further includes receiving, by a camera of the A/V device, imagedata of an object associated with the vehicle. The object may beidentified based on comparing the received image data of the object withpreviously stored image data of the object. Other methods of identifyingobjects are also described herein. The process further includestransmitting an actuation command to an actuator of the garage door tocause the garage door to close based on identification of the object.For example, if an A/V device identifies a license plate of a vehiclethat has left the garage, an actuation command may be transmitted toclose the garage door. In some embodiments, an electronic device (e.g.,the client device 214, 216, 230) may be communicated with before closingthe garage door. For example, the system may determine based oninformation from a client's smartphone or from a client vehicle devicethat a vehicle and/or client has left the garage. That information mayinclude geolocation data. A determination may also be made that thevehicle is gone based on other information related to a vehicle and/or aclient device. For example, if a client device is moving away from agarage at a speed, cadence, or other factor that is typical of movementof a vehicle driving forward on a road (e.g., highway, street, etc.),the system may assume that the client is in the vehicle and that thevehicle is outside of the garage. In this way, the system may determinethat it is likely that the vehicle is outside of the garage withoutusing geolocation or image data. In some embodiments, the system may usethis determination along with geolocation and/or image data to increasea level of confidence that a vehicle is outside of a garage.

FIG. 12 is a flowchart illustrating an example process 1200 formonitoring the movement of objects inside of a room, according tovarious aspects of the present disclosure. At block 1202, image data ofan area within the garage is received from an A/V recording andcommunication device. The area may be a predefined area, such as aspecific area on the floor, a shelving unit (e.g., the shelves 140), orany other area within a garage. For example, the backend server 224 mayreceive, using the communication module 504, image data 406 generated bythe A/V device 210 of the area of the garage. As described herein, areasother than the inside of a garage may also be monitored according to theprocess 1200 (e.g., other rooms in a building, an outside area, etc.).The image data 406 may depict objects within a field of view of the A/Vdevice 210 (e.g., a field of view of the camera 314 and/or a field ofview of the motion sensor(s) 326). In some examples, the backend server224 may receive the image data 406 from the A/V device 210, the hubdevice 202, and/or the client device 214, 216, 230. In some examples,the backend server 224 may receive the image data 406 based on the A/Vdevice 210 detecting motion within the line of sight of a camera and/ormotion sensor(s). The area of the garage may be an whole area inside agarage or may be part of a garage. In various embodiments, multiple A/Vdevices may be used to monitor different areas and/or to provide higherconfidence levels of monitoring one or more areas in common. The areamonitored may be, for example, the area of a garage monitored by thesecurity camera 142 of FIGS. 1B and 1C. The area monitored may be, forexample, the area monitored by the security camera 118 of the garage 114of FIG. 1A.

At block B1204 one or more objects are identified within the monitoredarea of the garage. The one or more objects may be identified by thevarious processes, methods, and/or algorithms described herein (e.g., anobject recognition algorithm). In addition to identifying discreteobjects within the monitored area, the system may identify what thoseobjects are (e.g., a vehicle, a crate, a lawnmower, a box, a barrel,etc.). The system may also identify a location within the garage (and/orwithin the monitored area of the garage) associated with each of the oneor more objects. In this way, an inventory may be created and storedthat includes objects in a garage. In some embodiments, images of theobjects in the garage may be timestamped or saved along with time anddate information (e.g., as metadata). Other information stored withand/or superimposed over an image may include the object identitiesdetermined from the image data, locations of the objects, etc. Thisinformation may, for example, be stored as identification dataassociated with the predefined area and the recognized one or moreobjects. This identification data may be stored as part of the garageinventory data 422 as described herein.

At block B1206, movement information is determined for each identifiedobject of the one or more objects. For example, movement information mayinclude information about whether any of the identified objects havebeen moved, for example by comparing a current timestamped locationassociated with the object with a previous timestamped locationassociated with the object. In another example, movement information maybe determined by comparing image data from a first time with image datataken at a second time. In another example, movement information may bedetermined based on identified movement within the garage. Movementinformation may be stored as part of the garage inventory data 422 asdescribed herein.

In various embodiments, alerts may be transmitted to a client devicewhen the object is determined to have been moved. For example, alertsmay be transmitted from the A/V device 210, the backend server 224, thesmart-home hub device 202, or another device to the client device 214,216, 230. In some embodiments, movement of an object meets a particularcriterion in order to trigger an alert. For example, an alert may bebased on a determination that an object is no longer present within thegarage and/or within the monitored area. In another example, an alertmay be based on the type of object that has been identified. Forexample, expensive identified items like vehicles, lawnmowers, skis,etc. may trigger alerts, while small boxes or other items in a garagenot identified or designated by a client as valuable may not triggeralerts. In another example, an alert may be triggered based on thepresence and/or absence of a client device. If a client device islocated in or around the garage, movement of objects may not triggeralerts. However, if a client device is not present or nearby whenobjects are moved, an alert may be triggered. In another example, alertsmay be triggered based on recognition of persons in and around thegarage. For example, facial recognition technology may be used torecognize a person in or around the garage. If the person is authorizedand/or known, movement of objects may not trigger an alert, whilemovement when no recognized persons are present may trigger an alert.Regardless of whether an alert is triggered in any of these scenarios,movement of objects may still be recorded as part of the garageinventory data 422, including any persons or client devices present,what objects have moved, what objects have been removed, etc.

In various embodiments, the system may determine if the movement of anobject is authorized. The determination of whether an object movement isauthorized may be similar to the criteria for whether to send an alertregarding object movement discussed above. If movement of an object isnot authorized, the system may transmit an audio command to a speaker ofan A/V device, a client device, a smart-home hub device, etc. includingan instruction to return a moved object to its previous location. Theinstruction may instruct a person to put an object back in a specificlocation that was previously determined by a client. The instruction mayinstruct a person to put an object back in a specific location thatis/was determined based on historical image data of the object and whereit is typically placed/located within a garage. The instruction may alsobe a generic instruction to put the object back without specifying aparticular location where the object must be put back.

At block B1208, whether a new object is moved into the area isdetermined. Similar to how objects are identified in a first instance,new objects (e.g., the crate 150 of FIG. 1C) may be identified as addedto a monitored area. Information about new objects may be reflected inthe garage inventory data 422. Information about new objects may includea time the objects were added, what type of object was added, a personidentified as present when the object was added, a client device presentwhen the object was added, a location of the added object.

Information reflected in the garage inventory data 422 may also includepoints of ingress/egress through which tracked objects moved. Forexample, if an object is removed from the garage, an A/V device maycapture data that indicates the object was taken out of the garagethrough a garage door opening. An object may also be taken out through awindow or doorway of the garage. Similar information may be determinedfor how objects are added to the garage. Information about points ofingress/egress may be saved in and reflected in the garage inventorydata 422. In various embodiments, location information for trackedobjects in the garage may be determined based on a particular planewithin the garage.

FIG. 13 is a flowchart illustrating an example process 1300 forproviding guest authorization to access a garage, according to variousaspects of the present disclosure. At block B1302, authorizationinformation relating to a vehicle is received. The vehicle may be, forexample, a guest vehicle to whom a client that controls a garage wishesto grant access to. For example, the backend server 224 may receive,using the communication module 504, the authorization information fromthe hub device 202 and/or the client device 214, 216, 230.

At block B1304, an indication of a predetermined number of times thevehicle may access a garage is received. The predetermined number oftimes may be received from a client device, where the client inputs thenumber of times they wish for the guest vehicle to be able to access thegarage. In various embodiments, the access may be granted for apredetermined period of time instead of a predetermined number of times.In another example, the guest vehicle may be able to access the garagefor a predetermined number of times, as long as those times are within apredetermined period of time. In another example, the predeterminednumber of times and/or amount of time during which a guest may access agarage is a default setting rather than set by the client.

At block B1306, a vehicle is detected within an area about the garage(e.g., similar to block B802 of FIG. 8). At block B1308, the vehicle isdetermined to be an authorized vehicle based on the authorizationinformation. The authorization information may include information aboutthe guest vehicle such as a license plate number, make, model, color, orany other aspect of a vehicle used to identify a vehicle as describedherein. The process to determine that the vehicle is an authorizedvehicle may be similar to any or all of blocks B804, B806, B808, and/orB810 as described herein.

At block B1310, the vehicle is determined to have accessed the garageless than the predetermined number of times. If the vehicle had alreadyaccessed the garage equal to or more times than the predetermined numberof times, access to the garage is denied (e.g., the garage door does notopen). In various embodiments, a predetermined amount of time may beused instead of or in addition to a predetermined number of times that aguest vehicle is permitted to access a garage. In such embodiments,block B1310 may also or instead determine whether the vehicle isattempting to access the garage within the predetermined amount of time.If not, access is denied. If access to the garage is not denied, theprocess 1300 proceeds to block B1312. At the block B1312, an actuationcommand is transmitted to an actuator of a garage door to cause thegarage door to open. In this way, a homeowner or other type of clientmay authorize guests to access their garage by entering informationabout the guests' vehicle into their client devices. Then when the guestvehicles approach the garage, the garage door is opened based on theembodiments described herein to provide access to the garage.

FIG. 14 is a flowchart illustrating an example process 1400 forproviding scannable codes for authorizing access to a garage, accordingto various aspects of the present disclosure. At block B1402, a requestfor a scannable code that provides temporary, limited time, or otherlimited access to a garage (e.g., for a guest) is received. For example,the backend server 224 may receive, using the communication module 504,from the A/V device 210, the hub device 202, and/or the client device214, 216, 230. Scannable codes may be an NFC code, Bluetooth code, QRcode, bar code, RFID code, or any other type of scannable code, whethervisually or electronically scannable.

At block B1404, the scannable code is sent to an electronic device. Thescannable code is sent to an electronic device so that the scannablecode may be presented using the electronic device to permit access tothe garage (e.g., cause the garage door to open). The scannable code maybe sent to the client device that requested the code, and the code maythen be sent onto another electronic device that allows the possessor ofthe code to access the garage. In various embodiments, the scannablecode may also be sent to an electronic device that did not request thecode. For example, the original request for the code may also includeinformation about an electronic device (e.g., an email address, phonenumber, other identifying information) to whom the scannable code shouldbe sent.

At block B1406, the scannable code is scanned in an area about thegarage. As described herein, scannable codes may be different types ofcodes and may be scanned in different ways. For example, an electronicscannable code may be scannable by using BLE, NFC, RFID, or other typeof short range communication. A visually scannable code (e.g., a barcode, a QR code) may be scannable by a camera or other sensor of an A/Vdevice and/or keypad.

At block B1408, the scannable code is determined to be valid to provideaccess to the garage. For example, the backend server 224 may receiveinformation relating to a code scanned by the A/V device 210, and thebackend server 224 may determine that the code is valid and send anactuation command to an actuator of the garage door (either directly orvia another device such as the A/V device 210) as shown at block B1410.

Visually scannable codes may be presented to be scanned in differentways. For example, a scannable code may be presented on the screen of anelectronic device so that it may be visually scanned. Paper with thecode printed thereon may be used (e.g., a sticker), and affixed to a carwindow, bumper, or other part of a car.

In various embodiments, facial recognition technology may be usedinstead or in addition to the scannable codes described above. That is,a face of a person may be used as a visual scannable code. In suchembodiments, a client device may capture with a camera one or moreimages of a guest to whom the client would like to grant access(including, e.g., an image of the face of the guest). Using facialrecognition, the face of the guest itself is the scannable code. Invarious embodiments, other biometric information about a person may beused as a scannable code. In contrast to the process 1400, the scannablecode (e.g., an image of a face) is provided by the client device, ratherthan the client device merely requesting a scannable code. The guest maythen present their face to a camera or other sensor in an area about thegarage. If the persons face is valid, access to the garage is granted(e.g., an actuation command is sent to the actuator of the garage door).

In various embodiments, scannable codes may be used to allow a deliverydriver into a garage to place a package for delivery. For example, thescannable code may be part of a label on a package that is delivered.The scannable code may permit access to leave the package inside thegarage. A scanned code may also trigger a notification to a clientdevice that the package has been delivered. A notification to a clientdevice may also be generated in response to an A/V device detecting thatthe package has been left in the garage (e.g., using internal garagemonitoring as described herein such as in FIG. 12). In variousembodiments, scannable codes as described herein may cause other signalsto be sent, functions to be performed, etc. For example, a particularscannable code may cause the garage door to close after a predeterminedamount of time (e.g., for a package delivery where the door does notneed to be open very long). The predetermined amount of time may also bebased on the number of packages to be delivered. In another example, thenumber of packages to be delivered may be known, and the scannable codecauses the garage door to stay open until the correct number of packageshave been placed inside the garage (and detected by A/V device(s)) andthe delivery driver is detected to be out of the garage. In anotherexample, the scannable code may cause an alert to be sent to a securitysystem or security monitoring service or may cause a security system tobe disarmed and armed again after the delivery is complete. In anotherexample, the scannable code may cause certain lights to be turned on oroff (e.g., in the garage).

FIG. 15 is a flowchart illustrating an example process 1500 forproviding a delay in closing a garage door to allow a person to exit agarage, according to various aspects of the present disclosure. At blockB1502, an instruction is received to close a garage door of a garage.This instruction may come from a standard garage door opener remote, abutton press of a keypad (e.g., the keypad 156), a button press of agarage door control interface (e.g., the garage door control interface632), a voice command, or any other method. In an alternativeembodiment, a device of the system (e.g., a device of FIG. 2) may notreceive the instruction to close the garage door. Instead, a device ofthe system may determine that the garage door has begun to close. Forexample, a camera and/or sensor(s) of the A/V device 210 or any otherdevice may capture data (e.g., image data) that indicates the garagedoor is closing. In various embodiments, a button may be pressed by aperson in the garage, such that the instruction to close the garage dooris sent directly to the garage door opener. In such embodiments, theinstruction may not be received by any of the A/V device 210, the clientdevices 214, 216, 230, the smart hub 202, the backend server 224, etc.Rather, the system may determine, based on image data, information froma garage door opener system, and/or elsewhere that an instruction toclose the garage door has been sent to the garage door opener.

At block B1504, a presence of a person within the garage is determinedfrom image data of an inside of the garage. If a person is in thegarage, at block B1506 the person is moving toward the garage door isdetermined from the image data. If the person is not moving toward thegarage door (e.g., if they are still, if they are moving in directionaway from the garage door such as toward a different doorway of thegarage), the system may not be concerned about providing time for theperson to get out of the garage before the garage door closes. Forexample, if a person has just parked her vehicle in a garage attached toa house, that person may then walk into the house from the garage.Accordingly, the system determines whether the person is walking towardthe garage door (or an opening of the garage for the garage door) todetermine whether to give the person more time to get out of the garagebefore closing the garage.

In some embodiments, the system may use other factors for context whendetermining whether a person is trying to exit out a garage openingafter an instruction to close the garage door is received. For example,the system may analyze typical movement patterns of people within thegarage, what objects are present or not present, and movement detectedin the time preceding a person trying to leave through a garage door asits closing. For example, a person may commonly try to run out of thegarage shortly after a car has pulled out of the garage (e.g., a familyalready in the car waiting for one member of the family in the drivewaybefore leaving). In another example, the system may recognize that aperson often tries to run out after closing a garage door after placinga lawnmower in the garage. By using movement and/or the presence/absenceof objects as context, the system may more accurately determine if aperson is trying to run out of the garage.

In various embodiments, the system may determine that a person is tryingto run out of the garage based on a sensor in a garage door opening,such as the sensor 160 of FIGS. 1D and 1E. In such an embodiment, if thesensor is tripped while the garage door is moving (or is about to move),the system may determine that a person is trying to run out of thegarage. In this example, the system may wait until the sensor 160returns to a normal state (e.g., does not sense the presence of anyonein the opening of the garage 152) before closing the garage door (orfinishing closing the garage door).

At block B1508, the garage door is determined to have already startedclosing. If the garage door has started closing (Yes) the process 1500proceeds to block B1510. If the garage door has not started closing (No)the process 1500 proceeds to block B1516.

At the block B1510, a stop actuation command is transmitted to anactuator of the garage door to cause the garage door to stop moving.This allows for more time for the person moving toward the garage doorto get out of the garage before the door closes. At the block B1512,that the person has moved outside of the garage is determined (e.g., isclear of the garage door, is clear of the opening 116 of the garage 114,etc.). At block B1514, a resume actuation command is transmitted to theactuator of the garage door to cause the garage door to continue toclose. Once the person is clear of the garage door and completelyoutside a garage, the resume actuation command is sent so that thegarage door may finish closing.

At the block B1516, transmission of an actuation command to an actuatorof the garage door to cause the garage door to close is delayed. Inother words, if the garage door has not started closing and the systemdetermines that a person is going to try to get out of the garage beforethe garage door closes, the system may delay sending the actuationcommand until the person is out of the garage and clear of the garagedoor. At block B1518, it is determined that the person has moved outsideof the garage (e.g., is clear of the garage door, is clear of theopening 116 of the garage 114, etc.). At block B1520, the actuationcommand is transmitted to the actuator of the garage door to cause thegarage door to close.

In various embodiments, the system may also broadcast an audio messagerelating to a delay in closing a garage door (whether the garage doorhas started closing or not). This may be helpful for a person moving outof a garage because the person may be concerned that the garage will notactually close once they are outside of the garage. For example, withrespect to the scenario where the garage door starts closing before theperson gets out of the garage (e.g., blocks B1510, B1512, B1514), thesystem may play an audio message over a speaker in the garage or on aclient device instructing the person that the garage door closing hasbeen delayed until the person gets outside of the garage. In anotherexample, with respect to the scenario where the garage door startsclosing after the person gets out of the garage (e.g., blocks B1516,1518, 1520), the system may play an audio message over a speaker in thegarage or on a client device instructing the person that the garage doorwill begin closing after the person is outside of the garage.

In various embodiments, various devices may be used to determine whenthe person moves outside of the garage. For example, referring to FIG.1A, a combination of the security camera 118 and the light camera 126may be used to determine that the person has left the garage. In thisexample, image data captured by the security camera 118 may be used todetermine that a person is trying to move out of the garage 114, whileimage data captured by the light camera 126 is used to determine/confirmthat the person has moved outside of the garage 114. In another example,referring to FIGS. 1D and 1E, the sensor 160 installed at a garage dooropening may also be used to help determine that the person has movedoutside of the garage 152. Specifically, image data from an A/V devicemay be used to determine that person is moving toward a garage door(e.g., block B1506). A tripping of the sensor 160 may indicate (or helpindicate in combination with other information such as image data) thatthe person has passed through the opening of the garage 152 and hasmoved outside the garage (e.g., blocks B1512 and B1518).

In various embodiments, A/V devices monitoring the inside of a garagemay be used to determine that a person left the garage through a doorother than the garage door (e.g., to a yard, to the inside of a house).Upon determining that the person has left the garage, the system maydetermine that the garage door can be closed.

FIG. 16 is a flowchart illustrating an example process 1600 for changingthe state of a garage door based on vehicle status information,according to various aspects of the present disclosure. At block B1602,vehicle status information is received from a computing deviceassociated with a vehicle. The computing device may be the client device230, an on-board diagnostic (OBD) computing device, or other type ofcomputing device that measures or has access to vehicle statusinformation. In one example, the computing device is the client device230 and accesses the vehicle status information via the OBD interface634.

At block B1604, a state of a garage door of a garage is determined. Forexample, the system may determine whether a garage door is open, closed,or in a state somewhere between open and closed. In various embodiments,the system may also determine whether the garage door is moving (e.g.,closing, opening). The state of the garage door as determined here maybe used along with the vehicle status information to determine if thestate of a garage door should be changed (e.g., send an actuationcommand to an actuator of the garage door).

At block B1606, a command to an actuator of the garage door to cause thestat of the garage door to change is transmitted based at least in parton the vehicle status and the state of the garage door. For example, avehicle may pull into a garage. The vehicle status information mayindicate when the vehicle has been shut off (or if the vehicle is stillrunning). If the garage door is open from the vehicle pulling in, thesystem waits until the vehicle status information indicates that the caris turned off to close the garage door. This may help, for example,prevent buildup of emissions from the vehicle in the interior of thegarage. In another example, the vehicle status information may indicatethat a vehicle parked in a garage has been started/turned on. If thegarage door is closed or not fully open, the system may send theactuation command to open the garage door based on the assumption thatwhoever started the car will want to exit the garage soon. This too mayhelp prevent buildup of emissions from the vehicle in the interior ofthe garage.

In various embodiments, different information than the vehicle statusinformation from an electronic device of a vehicle may be used todetermine vehicle status information. For example, an audio sensor suchas the microphone 328 of the A/V device 210 may be used to hear enginenoises and determine whether a car is turned on or off. In otherexamples, image data may be used to determine if a car is running ornot. In these embodiments, the system may determine whether a vehicle isrunning or not, which may be used to determine whether a state of agarage door should be changed.

In another example, a garage door may be opened (or start opening) as avehicle approaches a garage. However, the vehicle may stop in thedriveway and be shut off (e.g., the driver is just planning to park thecar in the driveway rather than the garage). In this scenario, thegarage may be closed once the car is shut off, as the vehicle is notlikely to be moved into the garage when it is shut off.

In various embodiments, the vehicle status may be used for otherpurposes, such as alerts or audio messages sent to the client device214, 216, 230, the smart hub device 202, the A/V device 220, and/or anyother device. For example, if a vehicle should be serviced (e.g., tirepressure adjusted, oil change, etc.), needs gas, or any other itemrelated to the vehicle status information, a text alert, audio message,or other sort of notification may be sent to notify a client.

FIG. 17 is a flowchart illustrating an example process 1700 for using avehicle location to determine whether a garage door should be closed,according to various aspects of the present disclosure. At block B1702,location data is received indicating a location of a vehicle. Thevehicle is associated with a garage. For example, the vehicle may belongto an owner or occupier of a home associated with the garage. Thelocation data may be geolocation data from, for example, the clientvehicle device 230.

At block B1704, a distance between the location of the vehicle and thegarage is determined to exceed a predetermined threshold. That is, thesystem determines if the vehicle is a certain distance or greater awayfrom the garage. At block B1706, a garage door of the garage isdetermined to be open. At block B1708, a command to an actuator of thegarage door to cause the garage door to close is transmitted based atleast in part on the determination that the garage is in an open stateand the determination that the distance between the location of thevehicle and the garage exceeds the predetermined threshold. In this way,the system may determine that a door should be closed when a vehicle hasmoved or is a certain distance away from the garage. This may bevaluable, for example, when a user prefers to send actuation commands totheir garage door manually, but may forget to do so in certaininstances. In various embodiments, an alert may be generated and sent toa client device, hub device, and/or a security monitoring service when avehicle moves or is a predetermined distance from the garage and thegarage door is open (whether or not an actuation command isautomatically transmitted or not).

FIG. 18 is a flowchart illustrating an example process 1800 forcontrolling a garage door with a voice command, according to variousaspects of the present disclosure. At block B1802, a voice commandsignal associated with a garage door of a garage is received. The voicecommand signal is a signal generated when a voice command is spoken by aperson and sensed by an audio sensor, such as a microphone. For example,the voice command signal may be received from the A/V device 210 usingthe microphone 328. The voice command signal may also be received fromthe smart hub device 202, the client device 214, 216, 230, or any otherdevice. A person speaking the voice command may be in varying locations.For example, with reference to FIG. 1A, the person 110 is be in theliving area 102 and speak a voice command that is sensed by an audiosensor of the smart-home hub device 108 and/or a client device of theperson 110. In another example, the person 124 is in the garage 114 andmay speak a voice command that is sensed by an audio sensor of thesecurity camera 118, a client device of the person 124, and/or a clientvehicle device of the vehicle 122. In another example, the person 112 isin the driveway 132 and may speak a voice command that is sensed by anaudio sensor of the light camera 126, the video doorbell 104, a clientdevice of the person 112, and/or a client vehicle device of the vehicle130. In another example, a person in any one of the vehicles 122, 130,or 136 may speak a voice command that is sensed by an audio sensor of aclient vehicle device of the respective vehicle.

At block B1804, an electronic device associated with an authorizedvehicle or authorized person is determined to be present in an areaabout the garage. This may be used to ensure that a voice command iscoming from a party authorized to control a garage door. Determiningthat the electronic device is present may be done in various ways. Forexample, the presence of the electronic device may be verified throughelectronic communication, such as short-range communications like NFC,BLE, Wi-Fi, or other short range electronic communication protocols. Inanother example, the presence of an electronic device may be determinedfrom image data (e.g., that the device is visible in image data capturedby a device in or near a garage).

In various embodiments, other methods may be used to determine that avoice command signal is valid and comes from a party authorized tocontrol a garage door. For example, a voice command signal may beconsidered to be authorized if it comes from a known client device(e.g., the client device 214, 216, 230). In another example, a voicecommand signal may be considered authorized if it comes from a devicethat should be in a secure area (e.g., non-public, or not easilyaccessible), such as the smart-home hub device 108 in the living area102, the security camera 118 in the garage 114, the security camera 142in FIGS. 1B and 1C, the client vehicle device 230, etc. In theseexamples, the assumption is that if a person has access to the area inwhich these devices are located (e.g., inside a home, garage, vehicle),then the voice command is authorized. In various embodiments, a personmay also be able to arm and/or disarm a security system using voicecommands, and those voice commands may be determined to be authorized insimilar ways. In various embodiments, the system may not make separatedeterminations that a voice command signal is authorized.

At block B1806, a command to an actuator of the garage door to cause thegarage door to move is transmitted based at least in part on the voicecommand signal. For example, a user may cause the garage door to open,close, stop moving, start moving, etc. with a voice command (e.g.,“close the garage door,” “open the garage door,” “stop,” etc.). Invarious embodiments, voice commands may be used to control other aspectsdescribed herein, such as smart-home hub devices (e.g., the hub devices202), the sensors 204, the automation devices 206, the virtual assistant(VA) device 208, the A/V devices 210, the client devices 214, 216, 230,or any other devices.

FIG. 19 is a flowchart illustrating an example process 1900 forauthorizing a single button press to change a state of a garage door,according to various aspects of the present disclosure. At block B1902,a transmission is received in response to a single button press of abutton associated with a garage door of a garage. For example, thebutton that is pressed may be a button of the keypad 156 of FIGS. 1D-1F.The button may also be mounted inside a garage. The button may also beintegrated in the A/V device 210, the client device 214, 216, 230, thehub device 202, or any other device. The button may be a physicalbutton, electronic sensor, a digital button displayed on a screen of anelectronic device, or any other type of button that allows for a clientto interface with.

At block B1904, an electronic device associated with an authorizedvehicle or authorized person is determined to be present in an areaabout the garage. This determination is made to authorize that thebutton press has been made by a party authorized to actuate a garagedoor. The authorization may be, in various embodiments, similar to anyof the embodiments discussed above with respect to block B1804. In someembodiments, the system may not authorize that a button press isauthorized. However, authorizing the button press provides addedsecurity. For example, on some keypads (e.g., the keypad 156) associatedwith garage doors, a user must input a long alphanumeric code in orderto authorize actuation of the garage door. In the process 1900, the usersimply presses one button, and the system authorizes the button pressthrough methods other than the long alphanumeric code. This saves theuser time, and reduces the chance a user would not be able to access thegarage because they forgot the long alphanumeric code. At block B1906, acommand to an actuator of the garage door to cause the garage door tomove is transmitted based at least in part on the determination that theelectronic device is present. Accordingly, a process for securelyactuating a garage door with a single button press is described herein.

In various embodiments, if a button press on a keypad or other deviceoccurs that is not deemed to be authorized, an alert may be generated.The alert may be sent to the client device 214, 216, 230, smart-home hubdevice 202, the security monitoring service 228, or any other device.

FIG. 20 is a flowchart illustrating an example process 2000 for using adistress code that opens a garage door and alerts a security monitoringservice, according to various aspects of the present disclosure. Atblock B2002, an alphanumeric code is received via a keypad at an areaabout a garage. The keypad may be, for example, the keypad 156 as shownin FIGS. 1D-1F. In various embodiments, the alphanumeric code may beentered through a device other a keypad, such as the A/V device 210, theclient device 214, 216, 230, the hub device 202, or any other device. Invarious embodiments, a different code may be used instead of analphanumeric code. For example, voice recognition may be used torecognize an audio code. Biometrics (e.g., face recognition) may be usedto recognize a particular biometric code.

At block B2004, a command to an actuator of a garage door of the garageto cause the garage door to open is transmitted in response to receivingthe alphanumeric code. At block B2006, an alert signal to a securitymonitoring computing device is transmitted further in response toreceiving the alphanumeric code. The security monitoring device may bepart of the security monitoring service 228. In this way, a particularcode entered by a user may open a garage door and send an alert to asecurity monitoring computing device. This allows a user to make an actsuch as opening a garage door and call for help using that act. If theuser is under distress and/or duress, this may be helpful and keep theuser safe until help arrives. In various embodiments, the code enteredmay be a distress code that does something other than open a garagedoor, such as control smart-home hub devices (e.g., the hub devices202), the sensors 204, the automation devices 206, the virtual assistant(VA) device 208, the A/V devices 210, the client devices 214, 216, 230,or any other devices.

A code that is enterable through a keypad may cause any of the actions,signals, alerts, actuations, etc. as the scannable codes describedherein. Similarly, a code for entering into a keypad may be generatedsimilar to the scannable codes described herein. For example, the keypadcodes may be used, generated, etc. in a similar way as described withrespect to FIG. 14 herein.

FIG. 21 is a flowchart illustrating an example process 2100 forcapturing an image in an opening of a garage, according to variousaspects of the present disclosure. At block B2102, a signal indicativethat a garage door motion sensor has detected motion in the area of anopening of a garage is received. The garage door motion sensor may, forexample, be the sensor 160 of FIGS. 1D and 1E. The garage door motionsensor may, for example, be the A/V device 210, or any other devicecapable of detecting motion.

At block B2104, a request to a camera to capture an image in response tothe signal is transmitted, where the camera has a field of view thatincludes at least a portion of the area of the opening of the garage. Atblock B2106, the image from the camera received. At block B2108, theimage is transmitted to a client electronic device. In this way, adevice (or multiple devices) may detect motion in a garage and transmitan image of the garage at the time motion is detected to a device, suchas the client device 214, 216, 230. In various embodiments, an imagerecognition and/or computer vision process may also analyze the image todetermine what caused the movement (e.g., person, animal, vehicle,etc.). The system may also determine if a sensor is broken ormalfunctioning (e.g., if nothing appears in the image that could havecaused the motion). In various embodiments, movement in the garage mayalso trigger an alert or alarm to be sent or activated to a device in ahouse, such as the smart-home hub device 108 in the living area 102 ofFIG. 1A.

FIG. 22 is a flowchart illustrating an example process 2200 fortransmitting a reminder message to complete a to-do list item based onvehicle location information, according to various aspects of thepresent disclosure. At block B2202, a to-do list item is received. Theto-do list item may be received from a smart-home hub device (e.g., thehub device 202), the automation devices 206, the virtual assistant (VA)device 208, the A/V devices 210, the client devices 214, 216, 230, orany other devices. A to-do list item may be manually or automaticallygenerated. For example, a smart refrigerator may identify that ahousehold is running out of (or is out of) milk. The smart refrigeratormay then create a to-do list item that more milk should be bought. Inanother example, a user may manually input a to-do list item into anelectronic device.

At block B2204, location information associated with a vehicle over aperiod of time is received. The period of time may be any period oftime. For example, the period of time may be one hour, twelve hours, oneday, two days, four days, one week, one month, etc. The period of timemay be a time after a first time when a vehicle leaves a garage and asecond time when the vehicle returns or is on its way to returning tothe garage. The period of time may be from a first time when the to-dolist item was received and a second time when a vehicle returns or is onits way to returning to the garage.

At block B2206, based at least in part on the location information, theto-do list item is determined to be not completed. For example, if theto-do list item was to buy milk, the location information may be used todetermine if the vehicle ever stopped at a grocery or other locationwhere milk could be purchased. If not, the system determines that theto-do list was not completed.

At block B2208, a reminder message to complete the to-do list item istransmitted to a client electronic device. For example, the remindermessage may be sent to the client vehicle device 230 to be played over aspeaker inside the vehicle as the vehicle is on its way home. Thereminder message may also be a text message that is displayed in thevehicle. In this way, the driver will conveniently be reminded of theto-do list before they get home. The reminder message may also be sentas the vehicle arrives in a garage, after the vehicle arrives in agarage, or at any other time. In another example, the reminder messagemay be sent to the client device 214, 216, the hub device 202, or anyother device in the form of an audio and/or text message.

FIG. 23 is a flowchart illustrating an example process 2300 fordetermining whether to change a state of a garage door based onatmospheric data measured in a garage, according to various aspects ofthe present disclosure. In various embodiments, sensors other thanatmospheric sensors may be used. For example, other environmentalsensors, noise or sound sensors, light sensors, or any other type ofsensor may be used as described herein. At block B2302, atmospheric datais received associated with atmospheric measurements taken by a sensorin a garage. For example, the sensor may be a carbon monoxide sensor,carbon dioxide sensor, thermometer, barometer, hygrometer, or any othertype of sensor that senses an atmospheric condition may be used.

At block 2304, a determination of whether a vehicle and/or person arepresent in the garage is made. This determination may be based on imagedata, or may be determined according to any other embodiments describedherein. At block B2306, a state of the garage is determined (e.g.,whether the garage is open, closed, somewhere in between, moving up,moving down, etc.). At block B2308, whether to transmit a signal to anactuator of the garage door to change the state of the garage door isdetermined based on the atmospheric data, the presence of the vehicleand/or the person, the state of the garage door, or any combinationthereof. Atmospheric sensors may be used to determine when to close oropen a garage door. For example, an oxygen, carbon dioxide, carbonmonoxide, or other type of air sensor may be used to sense air in thegarage and determine when to open or close the garage door. Such sensorsmay also be incorporated into a vehicle, and the sensor values may bereceived via an OBD interface from the vehicle (e.g., the OBD 634). Inanother example, sensors in the garage may be configured to open orclose the garage door to attempt to keep an atmospheric condition withinthe garage at, above, or below a predetermined level.

Accordingly, described herein are various embodiments for controlling agarage door that increase both the convenience of using a garage doorand the security of the garage door. For example, a garage door of thegarage 114 may open as the vehicle 130 approaches in the driveway 132.An A/V device such as a light camera 126 may capture image data of thevehicle 130, and then the system may determine aspects of the vehicle130 from the image data (e.g., license plate number/state, make, model,color, etc.) and determine if those aspects match a known vehicle. Ifthe aspects of the vehicle 130 match a known vehicle, the light camera126 (or another device) may communicates with a client device (e.g., theclient device 214, 216 (e.g., a smartphone) of a person in the vehicle130, the client vehicle device 230 of the vehicle 130) to determineauthorization information related to the vehicle. In variousembodiments, the system may scan a code or short range wireless deviceinstead of or in addition to communicating with a client device todetermine authorization information. If the authorization is successful,the system may send an actuation command to an actuator of the garagedoor to cause it to open. The light camera 126 also captures image dataof the vehicle 130 as it is leaving the garage 114. This image data isused to determine that the garage door may be safely closed. Once thevehicle is fully in view of the light camera 126, fully out of thedriveway 132, fully out of the opening 116, or some other criteria, anactuation command may be sent to the actuator of the garage door toclose the garage door.

Although several of the embodiments describe garage safety andconvenience features with respect to a residential context, the variousembodiments described herein may also be used in commercial, industrial,or other contexts. For example, the embodiments may be used in a largerresidential garage, commercial garages (e.g., vehicle serviceestablishments), industrial garages (e.g., manufacturing facilities,warehouses), etc. that have more than one door. A/V devices as describedherein may be used in these contexts to improve safety, convenience,profitability, and other factors. For example, a car serviceestablishment may use A/V devices to track vehicles moving in and out ofthe establishment. This may improve safety, as the A/V devices may beused to track movement within the establishment to prevent accidents.Alerts may be sent or broadcast to prevent a vehicle from hittingsomeone or something as described herein. Instructions on where toproperly park within a commercial or industrial establishment may alsobe broadcast as described herein.

Image data captured of vehicles in a commercial or industrial settingmay include license plate information. This information may be veryuseful to a commercial or industrial organization. For example, at a carservice garage (e.g., oil change business), A/V devices may pick uplicense plate information, which may be used to identify and/or track acustomer. If a customer has been to the business before, the vehicle andcustomer may be identified in the business's system with just thelicense plate information. This may cut down on data entry of thebusiness's employees. License plate information may be used forindustrial or other settings to track deliveries, vehicles in transit,etc. For example, if a semi-truck picks up goods at a manufacturingfacility to deliver to a warehouse or other location, image data from anA/V device may provide automated, real time updates to a system of whena truck leaves and arrives at a location, how long a truck has to waitto be loaded and/or unloaded, how long it takes to load and/or unloadthe truck, etc. Similar performance metrics may be measured in acommercial setting (e.g., how long does it take to complete an oilchange, how long is an average customer wait for an oil change, etc.).

FIG. 24 is a functional block diagram of a client device 2402 on whichthe present embodiments may be implemented according to various aspectsof the present disclosure. The client device(s) 214, 216, 230 describedwith reference to FIG. 2 may include some or all of the componentsand/or functionality of the client device 2402. The client device 2402may comprise, for example, a smartphone.

With reference to FIG. 24, the client device 2402 includes a processor2404, a memory 2406, a user interface 2408, a communication module 2410,and a dataport 2412. These components are communicatively coupledtogether by an interconnect bus 2414. The processor 2404 may include anyprocessor used in smartphones and/or portable computing devices, such asan ARM processor (a processor based on the RISC (reduced instruction setcomputer) architecture developed by Advanced RISC Machines (ARM).). Insome embodiments, the processor 2004 may include one or more otherprocessors, such as one or more conventional microprocessors, and/or oneor more supplementary co-processors, such as math co-processors.

The memory 2406 may include both operating memory, such as random-accessmemory (RAM), as well as data storage, such as read-only memory (ROM),hard drives, flash memory, or any other suitable memory/storage element.The memory 2406 may include removable memory elements, such as aCompactFlash card, a MultiMediaCard (MMC), and/or a Secure Digital (SD)card. In some embodiments, the memory 2006 may comprise a combination ofmagnetic, optical, and/or semiconductor memory, and may include, forexample, RAM, ROM, flash drive, and/or a hard disk or drive. Theprocessor 2404 and the memory 2406 each may be, for example, locatedentirely within a single device, or may be connected to each other by acommunication medium, such as a USB port, a serial port cable, a coaxialcable, an Ethernet-type cable, a telephone line, a radio frequencytransceiver, or other similar wireless or wired medium or combination ofthe foregoing. For example, the processor 2404 may be connected to thememory 2406 via the dataport 2412.

The user interface 2408 may include any user interface or presentationelements suitable for a smartphone and/or a portable computing device,such as a keypad, a display screen, a touchscreen, a microphone, and aspeaker. The communication module 2410 is configured to handlecommunication links between the client device 2402 and other, externaldevices or receivers, and to route incoming/outgoing data appropriately.For example, inbound data from the dataport 2412 may be routed throughthe communication module 2410 before being directed to the processor2404, and outbound data from the processor 2404 may be routed throughthe communication module 2410 before being directed to the dataport2412. The communication module 2410 may include one or more transceivermodules capable of transmitting and receiving data, and using, forexample, one or more protocols and/or technologies, such as GSM, UMTS(3GSM), IS-95 (CDMA one), IS-2000 (CDMA 2000), LTE, FDMA, TDMA, W-CDMA,CDMA, OFDMA, Wi-Fi, WiMAX, or any other protocol and/or technology.

The dataport 2412 may be any type of connector used for physicallyinterfacing with a smartphone and/or a portable computing device, suchas a mini-USB port or an IPHONE®/IPOD® 30-pin connector or LIGHTNING®connector. In other embodiments, the dataport 2412 may include multiplecommunication channels for simultaneous communication with, for example,other processors, servers, and/or client terminals.

The memory 2406 may store instructions for communicating with othersystems, such as a computer. The memory 2406 may store, for example, aprogram (e.g., computer program code) adapted to direct the processor2404 in accordance with the present embodiments. The instructions alsomay include program elements, such as an operating system. Whileexecution of sequences of instructions in the program causes theprocessor 2404 to perform the process steps described herein, hard-wiredcircuitry may be used in place of, or in combination with,software/firmware instructions for implementation of the processes ofthe present embodiments. Thus, the present embodiments are not limitedto any specific combination of hardware and software.

FIG. 25 is a functional block diagram of a general-purpose computingsystem on which the present embodiments may be implemented according tovarious aspects of the present disclosure. The computer system 2502 maybe embodied in at least one of a personal computer (also referred to asa desktop computer) 2504, a portable computer (also referred to as alaptop or notebook computer) 2506, and/or a server 2508 is a computerprogram and/or a machine that waits for requests from other machines orsoftware (clients) and responds to them. A server typically processesdata. The purpose of a server is to share data and/or hardware and/orsoftware resources among clients. This architecture is called theclient-server model. The clients may run on the same computer or mayconnect to the server over a network. Examples of computing serversinclude database servers, file servers, mail servers, print servers, webservers, game servers, and application servers. The term server may beconstrued broadly to include any computerized process that shares aresource to one or more client processes.

The computer system 2502 may execute at least some of the operationsdescribed above. The computer system 2102 may include at least oneprocessor 2510, memory 2512, at least one storage device 2514, andinput/output (I/O) devices 2516. Some or all of the components 2510,2512, 2514, 2516 may be interconnected via a system bus 2518. Theprocessor 2510 may be single- or multi-threaded and may have one or morecores. The processor 2510 execute instructions, such as those stored inthe memory 2512 and/or in the storage device 2514. Information may bereceived and output using one or more I/O devices 2516.

The memory 2512 may store information, and may be a computer-readablemedium, such as volatile or non-volatile memory. The storage device(s)2514 may provide storage for the system 2502 and, in some embodiments,may be a computer-readable medium. In various aspects, the storagedevice(s) 2514 may be a flash memory device, a hard disk device, anoptical disk device, a tape device, or any other type of storage device.

The I/O devices 2516 may provide input/output operations for the system2502. The I/O devices 2516 may include a keyboard, a pointing device,and/or a microphone. The I/O devices 2516 may further include a displayunit for displaying graphical user interfaces, a speaker, and/or aprinter. External data may be stored in one or more accessible externaldatabases 2520.

The features of the present embodiments described herein may beimplemented in digital electronic circuitry, and/or in computerhardware, firmware, software, and/or in combinations thereof. Featuresof the present embodiments may be implemented in a computer programproduct tangibly embodied in an information carrier, such as amachine-readable storage device, and/or in a propagated signal, forexecution by a programmable processor. Embodiments of the present methodsteps may be performed by a programmable processor executing a programof instructions to perform functions of the described implementations byoperating on input data and generating output.

The features of the present embodiments described herein may beimplemented in one or more computer programs that are executable on aprogrammable system including at least one programmable processorcoupled to receive data and/or instructions from, and to transmit dataand/or instructions to, a data storage system, at least one inputdevice, and at least one output device. A computer program may include aset of instructions that may be used, directly or indirectly, in acomputer to perform a certain activity or bring about a certain result.A computer program may be written in any form of programming language,including compiled or interpreted languages, and it may be deployed inany form, including as a stand-alone program or as a module, component,subroutine, or other unit suitable for use in a computing environment.

Suitable processors for the execution of a program of instructions mayinclude, for example, both general and special purpose processors,and/or the sole processor or one of multiple processors of any kind ofcomputer. Generally, a processor may receive instructions and/or datafrom a read only memory (ROM), or a random-access memory (RAM), or both.Such a computer may include a processor for executing instructions andone or more memories for storing instructions and/or data.

Generally, a computer may also include, or be operatively coupled tocommunicate with, one or more mass storage devices for storing datafiles. Such devices include magnetic disks, such as internal hard disksand/or removable disks, magneto-optical disks, and/or optical disks.Storage devices suitable for tangibly embodying computer programinstructions and/or data may include all forms of non-volatile memory,including for example semiconductor memory devices, such as EPROM,EEPROM, and flash memory devices, magnetic disks such as internal harddisks and removable disks, magneto-optical disks, and CD-ROM and DVD-ROMdisks. The processor and the memory may be supplemented by, orincorporated in, one or more ASICs (application-specific integratedcircuits).

To provide for interaction with a user, the features of the presentembodiments may be implemented on a computer having a display device,such as an LCD (liquid crystal display) monitor, for displayinginformation to the user. The computer may further include a keyboard, apointing device, such as a mouse or a trackball, and/or a touchscreen bywhich the user may provide input to the computer.

The features of the present embodiments may be implemented in a computersystem that includes a back-end component, such as a data server, and/orthat includes a middleware component, such as an application server oran Internet server, and/or that includes a front-end component, such asa client computer having a graphical user interface (GUI) and/or anInternet browser, or any combination of these. The components of thesystem may be connected by any form or medium of digital datacommunication, such as a communication network. Examples ofcommunication networks may include, for example, a LAN (local areanetwork), a WAN (wide area network), and/or the computers and networksforming the Internet.

The computer system may include clients and servers. A client and servermay be remote from each other and interact through a network, such asthose described herein. The relationship of client and server may ariseby virtue of computer programs running on the respective computers andhaving a client-server relationship to each other.

As used herein, the phrases “at least one of A, B and C,” “at least oneof A, B, or C,” and “A, B, and/or C” are synonymous and mean logical“OR” in the computer science sense. Thus, each of the foregoing phrasesshould be understood to read on (A), (B), (C), (A and B), (A and C), (Band C), and (A and B and C), where A, B, and C are variablesrepresenting elements or features of the claim. Also, while theseexamples are described with three variables (A, B, C) for ease ofunderstanding, the same interpretation applies to similar phrases inthese formats with any number of two or more variables.

The above description presents the best mode contemplated for carryingout the present embodiments, and of the manner and process of practicingthem, in such full, clear, concise, and exact terms as to enable anyperson skilled in the art to which they pertain to practice theseembodiments. The present embodiments are, however, susceptible tomodifications and alternate constructions from those discussed abovethat are fully equivalent. Consequently, the present invention is notlimited to the particular embodiments disclosed. On the contrary, thepresent invention covers all modifications and alternate constructionscoming within the spirit and scope of the present disclosure. Forexample, the steps in the processes described herein need not beperformed in the same order as they have been presented, and may beperformed in any order(s). Further, steps that have been presented asbeing performed separately may in alternative embodiments be performedconcurrently. Likewise, steps that have been presented as beingperformed concurrently may in alternative embodiments be performedseparately.

What is claimed is:
 1. A method of controlling a garage door to openusing an audio/video (A/V) recording and communication device, themethod comprising: detecting a vehicle within an area about the garagedoor; receiving, using a camera of the A/V recording and communicationdevice, image data representative of an object associated with thevehicle; comparing the image data representative of the objectassociated with the vehicle with previously stored image data;identifying the object based at least in part on the comparing of theimage data representative of the object with the previously stored imagedata; authenticating a mobile client device within the vehicle that isassociated with the object by wirelessly communicating with the mobileclient device; determining, based at least in part on the authenticatingof the electronic device within the vehicle, that the vehicle is anauthorized vehicle; and transmitting an actuation command to an actuatorof the garage door to cause the garage door to open.
 2. The method ofclaim 1, wherein authenticating the mobile client device comprises:receiving a set of attributes from the mobile client device within thevehicle; and comparing the set of attributes with previously storedattributes associated with a set of mobile client devices.
 3. The methodof claim 1, wherein the comparing of the image data representative ofthe object with the previously stored image data comprises an imagerecognition process.
 4. The method of claim 1, wherein the objectcomprises a license plate of the vehicle.
 5. The method of claim 1,wherein the object comprises a machine-readable optical label.
 6. Themethod of claim 1, further comprising, after the garage door is opened,establishing a wireless communication channel with a speaker associatedwith the vehicle.
 7. The method of claim 6, further comprisingtransmitting a signal to the speaker to cause the speaker to emit audioinstructions to a driver of the vehicle to move the vehicle to apredetermined position.
 8. The method of claim 6, further comprisingtransmitting a signal to the speaker to cause the speaker to emitaudible information about previous actuation commands, wherein theprevious actuation commands comprise a time of each actuation commandand an identification of a vehicle associated with each actuationcommand.
 9. The method of claim 1, further comprising, after the garagedoor is opened, providing, through a speaker of the A/V recording andcommunication device, an audio announcement comprising instructions tomove the vehicle to a predetermined position.
 10. The method of claim 9,wherein the audio announcement further comprises information aboutprevious actuation commands, wherein the previous actuation commandscomprise a time of each actuation command, and an identification of avehicle associated with each actuation command.
 11. The method of claim1, wherein the A/V recording and communication device comprises a garagedoor controller installed near the garage door.
 12. The method of claim1, wherein the A/V recording and communication device is connected to asecurity system.
 13. The method of claim 12, further comprisingreceiving information from the security system.
 14. The method of claim13, further comprising providing, via a speaker of the A/V recording andcommunication device, an audio announcement comprising the informationreceived from the security system.
 15. The method of claim 13, whereinthe information received from the security system comprises a number ofsecurity alerts generated by the security system while the vehicle wasoutside the garage.
 16. A method of controlling a garage door of agarage to close using an audio/video (A/V) recording and communicationdevice, the method comprising: determining that the garage door is open;detecting a vehicle within an area outside the garage; receiving, usinga camera of the A/V recording and communication device, image datarepresentative of the vehicle; analyzing the received image datarepresentative of the vehicle; determining, based at least in part onthe analyzing of the received image data representative of the vehicle,that the vehicle is moving away from the garage door; comparing thereceived image data representative of the vehicle with previously storedimage data representative of the vehicle; identifying the vehicle basedat least in part on the comparing of the received image datarepresentative of the vehicle with the previously stored image datarepresentative of the vehicle; and transmitting an actuation command toan actuator of the garage door to cause the garage door to close. 17.The method of claim 16, wherein comparing the received image datarepresentative of the vehicle with the previously stored image datarepresentative of the vehicle comprises a computer vision process. 18.The method of claim 16, wherein determining that the vehicle is movingaway from the garage door comprises performing a computer vision processon the received image data representative of the vehicle.
 19. The methodof claim 16, wherein determining that the vehicle is moving away fromthe garage door comprises comparing GPS location information received atthe A/V recording and communication device from an electronic device inthe vehicle.
 20. The method of claim 16, wherein the received image datarepresentative of the vehicle comprises image data representative of alicense plate of the vehicle.
 21. The method of claim 20, whereinidentifying the vehicle comprises identifying the license plate of thevehicle using an image recognition process on the image datarepresentative of the license plate of the vehicle.
 22. A method ofmonitoring a garage using an audio/video (A/V) recording andcommunication device installed within the garage, the A/V recording andcommunication device comprising a camera and at least one processor, themethod comprising: receiving, by the camera of the A/V recording andcommunication device, image data representative of a predefined areawithin the garage; performing, by the at least one processor, an objectrecognition algorithm on the received image data representative of thepredefined area within the garage; identifying a set of one or moreobjects within the predefined area within the garage and a locationassociated with each of the objects in the set; timestamping thereceived image data representative of the predefined area within thegarage as identification data associated with the predefined area withinthe garage and the set of one or more objects; and determining that oneof the objects in the set of one or more objects has been moved bycomparing a current timestamped location associated with the object witha previous timestamped location associated with the object.
 23. Themethod of claim 22, further comprising transmitting an alert to a clientdevice indicating that the one of the objects in the set of one or moreobjects has been moved.
 24. The method of claim 22, further comprisingproviding an audio command comprising an instruction to return the oneof the objects in the set of one or more objects to the previoustimestamped location.
 25. The method of claim 1, wherein theauthenticating of the mobile client device occurs while the vehicle iswithin the area about the garage door.
 26. The method of claim 16,wherein the determining that the vehicle is moving away from the garagedoor further comprises determining that a distance between the garagedoor and the vehicle exceeds a predetermined threshold distance.